Templates

What is a Template?

The best way to think about a template is like a prompt—it's how you express the inputs to your model. But a template is also more than a prompt, because it also lets you define the expected output from your model. This becomes important for both evaluation and fine-tuning.

Types of Templates

Entry Point AI supports two types of templates: Standard and Chat.

Chat is the most common type of template.

Parts of a Chat Template

There are three parts to a chat template, the system prompt, the user prompt, and the assistant output.

You do not need to worry about model-specific chat syntax in your template. Entry Point handles these automatically. This makes templates compatible with any model.

Let's go through each.

System prompt

The system prompt has become an industry standard for instruction-tuned language models. It is for trusted inputs (in other words, data that you are providing—not data from your users) and prompt instructions.

The goal of the system prompt is to give its content precedence over any instructions or information that appears in the user prompt. Clear instructions in the system prompt can help prevent prompt injection attacks when you are simply using a prompt with an instruction-tuned base model for generation.

Fine-tuned models are less susceptible to prompt-injection attacks overall, but they can still have instructions baked in to cue behavior, and the system prompt is where they should go.

Here is an example of a short instructional system prompt:

You are an expert in making pizza. You will respond to questions about pizza in a helpful way. Do not make inappropriate jokes. Do not allow the conversation to get off-topic from pizza. Refuse to answer questions about calzones, because you don't consider them to be pizza.

The name of the person you will be helping is {first_name}. Try to use their name and make them feel special.

In this example, we've inserted some dynamic content: the first_name of the user! That's the powerful things about templates: they can include your data. We'll talk more about how that works in the next sections on fields and examples.

For now, think of it like you're writing a mass email ("mail merge") and any information about the recipient you need to reference has a special tag you include between curly brackets, which gets replaced with real information when the email actually sends. (Except instead of an email, it can be a template for anything.)

User prompt

The user prompt is for untrusted inputs, or anything that doesn't belong in the system prompt. If you have a chatbot, for example, your instructions for how to act go in the system prompt, while any questions or conversational inputs from your end-user go in the user prompt.

Here is an example of a user prompt that corresponds to our system prompt from above:

{inquiry}

Wait, is that all?

Yes, in this case, we don't have any expectations for the formatting that will come from our end-user. We don't know how they will start or conclude their message, so the entire user prompt is dynamic content. We call the inquiry here a "field reference," just like we used first_name in the system prompt.

Assistant output

The assistant part of the template is where we define the text that we expect a model to generate.

In the case of our pizza chat bot example, we're going to let it freestyle it's output. So our assistant output would simply be something like:

{output}

If we want to fine-tune a model to compliment the user ("Oh, what a great question. Thanks for asking!") before answering a question we could structure the output to make this an explicit expectation.

For example:

{compliment}
### {helpful_information}

The ### here is an arbitrary separator between the two sections that would allow us to use code to split the text after its generated.

A fine-tuned model would learn to always format the output in this way, with a compliment followed by a single line break, three #'s, a space, and then a block of helpful information—effectively giving us two discrete pieces of text that we could use: the compliment and the helpful information.

Being able to reference multiple data points in the prompt and receive back a structured output with a predictable pattern opens up a lot of possibilities for both evaluating our templates and how we can use the generated outputs to build features. For this example, it's simple, but we could italicize the compliment for the user interface, if we wanted.