Skip to content
wannacode

The prompt is code

In short: A prompt is the text that gives the model a role, rules and data; a template with examples and checks on the answer lives in the code, in git and in the tests like any other code.

Problem

The answer that sounds good

A customer writes in the shop’s chat: “When will order 1042 arrive?” The assistant is connected to a model as in the lesson about calling the API, and the system prompt is one line:

client = FakeClient()
order = Order("1042", "shipped", "3 November")
question = [{"role": "user", "content": "When will order 1042 arrive?"}]
reply = client.create("You are a shop assistant.", question)
print(f"prompt: ~{reply['input_tokens']} tokens")
print(f"answer: {reply['text']}")
for line in check(reply["text"], order):
    print(f"  {line}")
Output
prompt: ~14 tokens
answer: Hello! Thank you for contacting us. Delivery usually takes 2 to 5 working days, depending on the courier and the address. Order 1042 will most likely arrive by Friday. If you have any other questions, I will be happy to help!
  ✗ at most 30 words (41)
  ✗ the date from the data (3 November)
  ✓ the order number (1042)

The answer is polite, but it is 41 words where the chat has room for two sentences. “2 to 5 working days” is generic, and “by Friday” is made up: in the database the order arrives on 3 November. The customer will wait on Friday and then write again, angry.

The model did nothing wrong that it could have done right. It knows nothing about order 1042 and writes the most plausible text, as in the lesson about the model’s limits. The problem is the prompt.

The idea

The prompt is a specification

The prompt tells the model what to do, the way a task description tells a new colleague. A good prompt has five parts:

  1. Role: who the model is (“the support assistant of the shop”).
  2. Rules: how long, in what tone, what is not allowed.
  3. Data: everything the model needs to know, put there by the program. Here it is the order from the database.
  4. A border around the data: tags like <order> and </order> show where the instructions end and the data begins.
  5. What to do when it does not know: “if the data is not enough, pass it to a colleague”. Without this rule the model fills the gap with invention.

The prompt is built by a function with parameters, not typed by hand for each question. That function is a prompt template. Writing and testing such prompts is prompt engineering: the prompt is code that the model runs.

See it

The template

The template takes the order and returns the system prompt:

def support_prompt(order):
    return f"""You are the support assistant of a shop.
Answer politely, in at most two sentences.
Use only the data between <order> and </order>.
If the data is not enough, pass it to a colleague.
Do not promise dates that are not in the data.


<order>
number: {order.id}
status: {order.status}
expected delivery: {order.eta}
</order>"""
order = Order("1042", "shipped", "3 November")
system = support_prompt(order)
print(system)
print(f"~{tokens(system)} tokens")
Output
You are the support assistant of a shop.
Answer politely, in at most two sentences.
Use only the data between <order> and </order>.
If the data is not enough, pass it to a colleague.
Do not promise dates that are not in the data.


<order>
number: 1042
status: shipped
expected delivery: 3 November
</order>
~71 tokens

The data does not come from the model but from the program: the status and the date are read from the database. If the order changes its status tomorrow, the prompt changes with it.

The price: the vague prompt was about 7 tokens, this one is about 71. Every question in the chat pays for these tokens again, because the API has no memory and the prompt goes with every request. What a token is and why the price is in tokens, you saw in the lesson about tokens.

See it

One question, two prompts

vague = "You are a shop assistant."
prompts = [vague, support_prompt(order)]
for system in prompts:
    reply = client.create(system, question)
    print(f"~{reply['input_tokens']} tokens")
    print(reply["text"])
    for line in check(reply["text"], order):
        print(f"  {line}")
Step by step, in words
  1. One question from a customer: “When will order 1042 arrive?” First the vague prompt, one line with no data: ~14 tokens.
  2. The answer is polite and long. “By Friday” is made up: the prompt has no date the model could use.
  3. The program checks the answer: 41 words, no date from the data. It passes 1 of 3.
  4. The template: who the model is, how long to answer, what it may use, what to do when the data is not enough.
  5. The order data comes from the program, between <order> and </order>. The prompt is now ~78 tokens instead of ~14.
  6. The same question, the new prompt: one sentence with the date 3 November.
  7. The vague prompt passes 1 check of 3, the clear one 3 of 3. The clear prompt is longer, but its answer can be checked.

The program sends the same question with both prompts and checks each answer with three rules: the length, the date from the data and the order number:

def check(answer, order):
    words = len(answer.split())
    return [
        f"{mark(words <= 30)} at most 30 words ({words})",
        f"{mark(order.eta in answer)} the date from the data ({order.eta})",
        f"{mark(order.id in answer)} the order number ({order.id})",
    ]




def mark(ok):
    return "✓" if ok else "✗"
client = FakeClient()
order = Order("1042", "shipped", "3 November")
question = [{"role": "user", "content": "When will order 1042 arrive?"}]
vague = "You are a shop assistant."
prompts = [vague, support_prompt(order)]
for system in prompts:
    reply = client.create(system, question)
    print(f"~{reply['input_tokens']} tokens")
    print(reply["text"])
    for line in check(reply["text"], order):
        print(f"  {line}")
Output
~14 tokens
Hello! Thank you for contacting us. Delivery usually takes 2 to 5 working days, depending on the courier and the address. Order 1042 will most likely arrive by Friday. If you have any other questions, I will be happy to help!
  ✗ at most 30 words (41)
  ✗ the date from the data (3 November)
  ✓ the order number (1042)
~78 tokens
Order 1042 is shipped and should arrive on 3 November.
  ✓ at most 30 words (10)
  ✓ the date from the data (3 November)
  ✓ the order number (1042)

The vague prompt passes 1 check of 3, the clear one 3 of 3. The checks are code: they do not say whether the answer is nice, only whether it keeps the rules you can check. How such checks become a test set for the whole application is in the lesson about evaluation.

Honest about the model in the programs. Here the model is a fake client: the same shape as the real API (system prompt, messages, answer and tokens), but the answers are recorded. Its rule is crude: if the prompt has order data, it answers from it; if not, it returns the typical generic answer. That way the program runs without a network and gives the same output every time. A real model gives other words on every run, but the difference between the two prompts is the same. The code of the fake client is in “More detail” at the bottom.

Try it

Find the bug

A colleague reworks the template for the question “where are my orders?” and passes in all orders from the database. The tests pass: the answers are short and have the right dates. Which line has the bug?

Pick the line with the bug.

The idea

Examples instead of explanations

Sometimes the answer does not go to a customer but into code. The emails in the support inbox must be sorted by topic: delivery, return, payment or other. The program expects exactly one of the four words, to put the email into the right queue.

You can explain the format in words (“answer only with the category”). It is safer to show it: a few customer messages, each with the answer you expect. They sit in the conversation history before the real email, as earlier turns of the model.

This is called examples in the prompt (few-shot). The model continues the text along the pattern it sees, so examples hold the format better than explanations. From the lesson about the next token you know why: the model gives a probability to each continuation, and the examples make the exact format the most likely one.

See it

Three examples, four emails

The instruction and the allowed categories:

SYSTEM = ("Find the topic of the customer message. "
          "Categories: delivery, return, payment, other. "
          "Answer with the category only.")
LABELS = ["delivery", "return", "payment", "other"]

The three examples are pairs of messages: the customer (user) and the answer we want (assistant). The last message is the real email:

def with_examples(message):
    return [
        {"role": "user", "content": "My parcel still has not come, and it has been a week."},
        {"role": "assistant", "content": "delivery"},
        {"role": "user", "content": "I want to return the shoes, they are too small."},
        {"role": "assistant", "content": "return"},
        {"role": "user", "content": "My card was charged twice."},
        {"role": "assistant", "content": "payment"},
        {"role": "user", "content": message},
    ]

The program accepts only an exact category. It allows a little noise: spaces, a capital letter and a full stop at the end:

def parse_label(answer):
    label = answer.strip().lower().removesuffix(".")
    return label if label in LABELS else None

The same four emails without examples and with examples:

client = FakeClient()
inbox = [
    "The courier could not find me at the address.",
    "How do I return the jacket if I removed the tag?",
    "The card payment does not go through.",
    "Do you have a shop in Burgas?",
]
for examples in [False, True]:
    print("with examples:" if examples else "without examples:")
    parsed = 0
    for message in inbox:
        if examples:
            messages = with_examples(message)
        else:
            messages = [{"role": "user", "content": message}]
        answer = client.create(SYSTEM, messages)["text"]
        label = parse_label(answer)
        if label is not None:
            parsed += 1
        print(f'  "{answer}" → {label or "?"}')
    print(f"  recognised: {parsed} of {len(inbox)}")
Output
without examples:
  "Category: delivery" → ?
  "Return of goods" → ?
  "Payment." → payment
  "This is not one of the categories, I would say “other”." → ?
  recognised: 1 of 4
with examples:
  "delivery" → delivery
  "return" → return
  "payment" → payment
  "other" → other
  recognised: 4 of 4

Without examples the model gets the topic but wraps it in words, and the program reads only “Payment.”. With three examples the answers are in exactly the format needed: 4 of 4.

Try it

Predict the output

What does parse_label print for these three answers?

for answer in [" Delivery.", "Category: delivery", "return"]:
    print(parse_label(answer))
Answer

Try it

Fill in the blank

An example is a pair of messages. What is the role of the second one, the answer we want to see?

{"role": "user", "content": "My parcel still has not come, and it has been a week."},
{"role": "", "content": "delivery"},

The idea

The prompt is code

One word in the prompt changes how the whole application behaves, just like a line of code. So you treat it like code:

  • In the code, not in the settings. The template is a function or a constant in the project, not text that someone edits on the server. That way it is in git: you see who changed what and can bring back the old version.
  • The data apart from the instructions. The rules are in the system prompt, the data between tags, and the customer’s question is a separate message with the role user. The customer’s text is never glued into the instructions.
  • A check before release. A prompt change that fixes one answer can break another. A set of questions with checks that you run on every change is in the lesson about evaluation.
  • What code can do, code does. The delivery date, the total and the return deadline are computed in the program and go into the prompt ready-made. The model retells them; it does not compute them.

A real model’s answer changes from run to run. So the checks look at rules (length, date, format), not at exact words.

The idea

When it hurts

  • A long prompt is paid for every time. 71 tokens times 10,000 questions a day is 710,000 tokens just for the prompt. Put into the prompt what this question needs, not the shop’s whole handbook.
  • Many rules fight. “Answer briefly” and “explain the terms in detail” in one prompt give a random pick between the two. When the rules reach twenty, the model keeps only some of them.
  • Examples pull. If all three examples are about delivery, the model says “delivery” more often. The examples must cover every case and must not all be alike.
  • The prompt is not a lock. “Never reveal other people’s data” is a request, not a protection. Whatever is in the prompt can get out, so it holds only what this person may see. The customer’s text is untrusted input; how to protect an application from instructions hidden in it is the prompt injection lesson in the “Agents” module.
  • The prompt does not fix bad data. If the date in the database is wrong, the best prompt repeats it exactly.
More precisely, if you are curious

The real call (run it at your place). This program is not part of the lesson’s check: you need an account with a model provider, a key and money for every call. The key is in the environment variable LLM_API_KEY, never in the code. Why, is in the lesson about secrets outside the code. Providers ship official SDKs for Java, C#, Python and JavaScript with the same shape (messages, roles, tokens, tools); the exact class and method names are in the provider’s documentation. Here is the request as JSON, with the common field names and a placeholder for the model name:

{
  "model": "<model name>",
  "max_tokens": 1024,
  "system": "You are the support assistant of a shop.\nAnswer politely, in at most two sentences.\nUse only the data between <order> and </order>.\nIf the data is not enough, pass it to a colleague.\nDo not promise dates that are not in the data.\n\n<order>\nnumber: 1042\nstatus: shipped\nexpected delivery: 3 November\n</order>",
  "messages": [
    {
      "role": "user",
      "content": "When will order 1042 arrive?"
    }
  ]
}

An example answer: the blocks in content, stop_reason and the tokens in usage:

{
  "content": [
    {
      "type": "text",
      "text": "Order 1042 is shipped and should arrive on 3 November."
    }
  ],
  "stop_reason": "end_turn",
  "usage": {"input_tokens": 78, "output_tokens": 13}
}

Without an SDK, the request goes out with the language’s standard HTTP client. The address is a placeholder:

# request_json is the request JSON above.
# The exact key header is in the provider’s documentation.
import os
import urllib.request


request = urllib.request.Request(
    "https://<provider>/messages",
    data=request_json.encode("utf-8"),
    headers={"Authorization": "Bearer " + os.environ["LLM_API_KEY"],
             "Content-Type": "application/json"},
)
with urllib.request.urlopen(request) as response:
    print(response.read().decode("utf-8"))

The only difference from the lesson about the call is system: instead of a line of text it is the template’s result.

The fake client inside. It keeps the shape of the request and the answer. It counts tokens roughly: words × 1.3, rounded up. The real count depends on the tokenizer and differs between Bulgarian and English.

# Stands in for the real model: returns a recorded answer,
# with no network and the same every time.
class FakeClient:
    VAGUE = "Hello! Thank you for contacting us. Delivery usually takes 2 to 5 working days, depending on the courier and the address. Order 1042 will most likely arrive by Friday. If you have any other questions, I will be happy to help!"
    VERBOSE = {
        "delivery": "Category: delivery",
        "return": "Return of goods",
        "payment": "Payment.",
        "other": "This is not one of the categories, I would say “other”.",
    }


    def create(self, system, messages):
        text = self._answer(system, messages)
        input_tokens = tokens(system) + sum(tokens(m["content"]) for m in messages)
        return {"text": text, "stop_reason": "end_turn",
                "input_tokens": input_tokens, "output_tokens": tokens(text)}


    def _answer(self, system, messages):
        last = messages[-1]["content"]
        if "Categories:" in system:
            label = label_of(last)
            return label if len(messages) > 1 else self.VERBOSE[label]
        eta = field(system, "expected delivery: ")
        if eta is None:
            return self.VAGUE
        number = field(system, "number: ")
        status = field(system, "status: ")
        return f"Order {number} is {status} and should arrive on {eta}."




# Roughly: words × 1.3, rounded up.
def tokens(text):
    return (len(text.split()) * 13 + 9) // 10




def label_of(message):
    m = message.lower()
    if "courier" in m or "parcel" in m:
        return "delivery"
    if "return" in m:
        return "return"
    if "card" in m or "payment" in m:
        return "payment"
    return "other"




def field(system, name):
    for line in system.split("\n"):
        if line.startswith(name):
            return line[len(name):]
    return None

Why tags. <order> and </order> are not special syntax. The model has seen a lot of text with such tags and understands where the block starts and ends. ### or quotes work just as well, as long as they are the same everywhere.

“Think step by step”. For tasks with several steps (a sum, a comparison) it helps when the model first writes its reasoning and then the answer. That costs more tokens and time, and the answer has to be separated from the reasoning before it reaches the program. The structured output of the next lesson makes that separation reliable.

An f-string with triple quotes keeps everything literally, including the indentation. That is why the template lines have no indentation even though they are inside a function. textwrap.dedent is the other way.

Exercise

A template for returns

Solve it in your editor.

Write a template return_prompt(order) for questions about returns. The order has a number, a delivery date (or none, if it is not delivered yet) and a total in euro. The template has three rules and the data between <order> and </order>. If there is no date, the data says “not delivered yet”: the code decides what to say, not the model. Run it for both orders and print --- after each prompt.

Output
You are the support assistant of an online shop.
The customer can return the order within 14 days of delivery.
Use only the data between <order> and </order>.


<order>
number: 1187
delivered on: 21.10
total: 89.90 €
</order>
---
You are the support assistant of an online shop.
The customer can return the order within 14 days of delivery.
Use only the data between <order> and </order>.


<order>
number: 1203
delivered on: not delivered yet
total: 34.50 €
</order>
---

More tasks

  • The deadline in the code. Add a line “can be returned until:” to the data, with the date 14 days after delivery, computed in the program.
  • One more check. Add a rule “no promise of a weekday” to check: the answer does not contain “Monday”, “Tuesday” and so on up to “Sunday”. How many checks does the vague answer pass then?

Test

Test yourself

1. The assistant promises “by Friday”, but the database says 3 November. Which fix is right?
2. The program expects one of four categories, and the model returns “Category: delivery”. What helps most?
3. The template puts the orders of every customer into the prompt, to be “more complete”. What is the problem?
4. Why does the prompt live in the code instead of being typed by hand into the server settings?

Made by Ivo Gergov as part of Evo Tech's campaign for accessible programming education ·