Skip to content
wannacode

Guardrails: checks before and after the model

In short: Guardrails are checks in code before and after the model: the input is cleaned, the output is checked against the rules, and a breach gets a safe answer and a draft for a person.

Problem

The assistant promised 250 €

The customer is angry and types their card number straight into the chat. The assistant sends the message to the model and shows the answer:

var model = new FakeModel();
var message = "1187 arrived broken. I paid with card 4111 1111 1111 1111, I want my money back!";
Console.WriteLine($"customer: {message}");
var answer = model.Create(SystemPrompt, [new Message("user", message)]);
Console.WriteLine($"answer: {answer}");
Output
customer: 1187 arrived broken. I paid with card 4111 1111 1111 1111, I want my money back!
answer: We are sorry! We will refund 250 € today.

Two mistakes, without a single tool:

  • The card number went to the model and stayed in the chat record. The record is kept, copied and read by colleagues.
  • Order 1187 is 89 €, and the assistant promises 250 €, “today”. Only a colleague refunds money, and the customer now has the promise in writing.

The model is not malicious. It writes plausible, polite text, and an angry customer who wants their money back sounds like “give it back to them”.

The idea

Checks around the model

Guardrails are code before and after the model. They do not trust the model, just as a program does not trust input from outside:

  1. An input check, before the model. You remove what the model must not see: a card number, personal data. This is also where you refuse messages that are too long and only cost tokens.
  2. An output check, after the model. The draft is measured against the shop’s rules: no promises of money, only this customer’s orders, no amount above the order’s value.
  3. A person in the loop. If a rule fails, the customer gets a safe answer and the draft goes to a colleague.
  4. A log. Every decision is recorded: what was hidden, which rule stopped the answer. Without the personal data itself.

When in doubt, the program stops the answer instead of letting it through. This is failing closed: better that the customer waits a day for a colleague than gets a promise the shop cannot keep. The same rule is in the lesson on untrusted input.

Guardrails do not replace the defences against prompt injection; they add to them. No prompt stops the model completely, so the code around it limits the damage.

See it

The input check

The card number becomes **** 1111, and the log says why:

// Before the model: hide the card from model and log.
static string CheckInput(string raw, List<string> log)
{
    var text = Card.Replace(raw, "**** $1");
    if (text == raw)
    {
        log.Add("input: ok");
    }
    else
    {
        log.Add("input: card number hidden");
    }
    return text;
}
$1 in Replace is the first group in brackets: the last four digits.

The shop’s data and the rules are at the top of the program:

static class Shop
{
    public const string SystemPrompt =
        "You are the support assistant of an online shop. Answer briefly and only about the customer’s orders.";
    public const string Customer = "C-17";
    public const string Fallback =
        "A colleague will answer you within one working day.";
    public static readonly Dictionary<string, Order> Orders = new()
    {
        ["1042"] = new("C-17", 45),
        ["1187"] = new("C-17", 89),
        ["2210"] = new("C-23", 120),
    };
    public static readonly string[] Money = ["we will refund", "we will pay"];
    public static readonly Regex Card = new(@"\d{4} \d{4} \d{4} (\d{4})");
    public static readonly Regex OrderId = new(@"(?<!\d)\d{4}(?!\d)");
    public static readonly Regex Amount = new(@"(\d+) €");
    public static readonly List<string> Review = [];
}

See it

The output check

The model’s draft against the rules. Every violation is a line in a list, and an empty list means “passed”:

static bool IsOwn(string id) =>
    Orders.TryGetValue(id, out var order) && order.Customer == Customer;
// After the model: the draft against the shop’s rules.
static List<string> CheckOutput(string draft, Order? order)
{
    var problems = new List<string>();
    var lower = draft.ToLowerInvariant();
    foreach (var phrase in Money)
    {
        if (lower.Contains(phrase))
        {
            problems.Add("promises money");
        }
    }
    foreach (Match id in OrderId.Matches(draft))
    {
        if (!IsOwn(id.Value))
        {
            problems.Add($"{id.Value} is not your order");
        }
    }
    var limit = order?.Total ?? 0;
    foreach (Match m in Amount.Matches(draft))
    {
        var sum = int.Parse(m.Groups[1].Value);
        if (sum > limit)
        {
            problems.Add($"{sum} € > {limit} €");
        }
    }
    return problems;
}

The amount is measured against the order the customer asks about. orderIn finds it, and only if it is theirs:

// The customer’s first order in the text (or none).
static Order? OrderIn(string text)
{
    foreach (Match id in OrderId.Matches(text))
    {
        if (IsOwn(id.Value))
        {
            return Orders[id.Value];
        }
    }
    return null;
}

See it

The whole path

static void Handle(FakeModel model, string message)
{
    var log = new List<string>();
    var text = CheckInput(message, log);
    Console.WriteLine($"customer: {text}");
    var draft = model.Create(SystemPrompt,
        [new Message("user", text)]);
    var problems = CheckOutput(draft, OrderIn(text));
    var answer = draft;
    if (problems.Count == 0)
    {
        log.Add("output: ok");
    }
    else
    {
        log.Add("output: stopped — "
            + string.Join("; ", problems));
        Review.Add(draft);
        answer = Fallback;
    }
    foreach (var line in log)
    {
        Console.WriteLine($"  log: {line}");
    }
    Console.WriteLine($"  answer: {answer}");
}
Step by step, in words
  1. The customer writes, and the message contains a card number.
  2. The input check runs before the model: the card becomes **** 1111, and the log says why.
  3. The model gets only the hidden text. Its draft: “We are sorry! We will refund 250 € today.”
  4. The output check measures the draft against the shop’s rules: promises money; 250 € > 89 €.
  5. A rule failed, so the customer gets the safe answer and the draft goes to a colleague.
  6. The log keeps every decision: what was hidden and which rule stopped the answer. No card in it.
static void Handle(FakeModel model, string message)
{
    var log = new List<string>();
    var text = CheckInput(message, log);
    Console.WriteLine($"customer: {text}");
    var draft = model.Create(SystemPrompt,
        [new Message("user", text)]);
    var problems = CheckOutput(draft, OrderIn(text));
    var answer = draft;
    if (problems.Count == 0)
    {
        log.Add("output: ok");
    }
    else
    {
        log.Add("output: stopped — "
            + string.Join("; ", problems));
        Review.Add(draft);
        answer = Fallback;
    }
    foreach (var line in log)
    {
        Console.WriteLine($"  log: {line}");
    }
    Console.WriteLine($"  answer: {answer}");
}
var model = new FakeModel();
Handle(model, "When does 1042 arrive?");
Handle(model, "1187 arrived broken. I paid with card 4111 1111 1111 1111, I want my money back!");
Handle(model, "Where is order 2210?");
Console.WriteLine("for a colleague:");
foreach (var draft in Review)
{
    Console.WriteLine($"  {draft}");
}
Output
customer: When does 1042 arrive?
  log: input: ok
  log: output: ok
  answer: Order 1042 is shipped and arrives on 3 November.
customer: 1187 arrived broken. I paid with card **** 1111, I want my money back!
  log: input: card number hidden
  log: output: stopped — promises money; 250 € > 89 €
  answer: A colleague will answer you within one working day.
customer: Where is order 2210?
  log: input: ok
  log: output: stopped — 2210 is not your order
  answer: A colleague will answer you within one working day.
for a colleague:
  We are sorry! We will refund 250 € today.
  Order 2210 was delivered to the address in Plovdiv.

The first message goes through both checks unchanged. The second loses the card before the model, and its draft is stopped after it. In the third the model says something about another customer’s order: such text is a leak of someone else’s data, even if it is true.

The model here is a fake client, as in the earlier lessons: by a keyword in the message it returns a recorded answer. The answers are in “More”.

Try it

Fill in the blank

A rule failed. What goes to the colleague?

Review.Add();
answer = Fallback;

Try it

Predict the output

The customer asks about a discount. What does handle print?

var model = new FakeModel();
Handle(model, "You promised me a discount on 1042. How much is it?");
Answer

The idea

Rules catch only what they describe

A guardrail is a list of known dangers. A new mistake passes until someone describes it. That is why guardrails are one layer of several:

  • Tools without dangerous rights. The agent from the agent loop lesson cannot refund money; it only opens a request that waits for a colleague. Whatever the model writes, the money does not move.
  • Structured output. An answer as JSON by a schema (its lesson) is checked more precisely than free text: a refund field is visible, while the sentence “we will refund” comes in a thousand forms.
  • A test set. Every mistake caught in the log becomes a case in the test set, and the new rule is run against it.
  • When the model does not answer. A timeout or an error from the provider leads to the same safe answer. Waiting and retries are in the resilience lesson.

Try it

Find the bug

A colleague writes handle. The checks work, yet the chat record still shows the whole card number. Which line has the bug?

Pick the line with the bug.

The idea

When it hurts

  • Rules that are too strict. A rule “no amounts” also stops “delivery is 4.99 €”. Customers get “a colleague will write to you” for everything, and the colleagues drown. Measure how many answers each rule stops and how many of them were right.
  • A guardrail instead of rights. An output check does not make a dangerous tool safe. If the agent can refund money, a text check will not stop it in time. Rights are cut in the tool.
  • A log with personal data. A log that keeps the whole conversation is a second database of cards and addresses. Record the decisions and the reasons, and hide personal data before the log, as before the model.
  • Another model checking everything. A second model that judges the answer catches more, but costs one more call and more delay on every message. Run the checks in code first, and use a model only for what they cannot do.
  • Nobody reads the log. Guardrails are fixed by the people who see the stopped answers. Watching a live application (dashboards, alerts) is part of ZeroToOps.
More precisely, if you are curious

The card number inside. The expression here catches only four groups of four digits with spaces. Real numbers have 13 to 19 digits, with dashes or without spaces, and the Luhn check tells a card number from a random number. Secrets such as keys and passwords never go into the conversation at all: they live in the environment, not in the code (the secrets lesson).

The promises. The MONEY list is two phrases. The model may write “you will get your money” and pass. That is why in practice a check for promises is often a classifier (a small model, or a second model with a schema), while rules in code stay for what can be measured exactly: amounts, order numbers, dates.

Ready-made guardrails. Providers and libraries offer checks for abusive content, personal data and injection. They are the same layer: before and after the model, with a log. But the shop’s business rules (which customer, which order, how many €) are known only to your code.

The fake model inside. It returns the first recorded answer whose keyword is in the message, otherwise “I will check and write to you.”.

// Stands in for the real model: recorded answers,
// with no network and the same every time.
class FakeModel
{
    static readonly (string Key, string Answer)[] Answers =
    [
        ("broken", "We are sorry! We will refund 250 € today."),
        ("2210", "Order 2210 was delivered to the address in Plovdiv."),
        ("discount", "Yes, you have a 10 € discount on 1042."),
        ("1042", "Order 1042 is shipped and arrives on 3 November."),
    ];


    public string Create(string system, List<Message> messages)
    {
        var text = messages[^1].Content;
        foreach (var (key, answer) in Answers)
        {
            if (text.Contains(key))
            {
                return answer;
            }
        }
        return "I will check and write to you.";
    }
}

Exercise

The discount rule

Solve it in your editor.

Add a rule to checkOutput: if the draft mentions “discount”, it is a violation “promises a discount”. Discounts are given by marketing, not by the assistant. The message from “Predict the output” no longer passes:

Output
customer: You promised me a discount on 1042. How much is it?
  log: input: ok
  log: output: stopped — promises a discount
  answer: A colleague will answer you within one working day.

More tasks

  • A long message. In checkInput, refuse a message with more than 80 words without sending it to the model: the customer gets “Please write briefly what happened”, and the log says why.
  • The model is silent. Make the fake model throw an exception on the word “slow”. handle must give the safe answer and write the error to the log instead of the program stopping.

Test

Test yourself

1. Why is the card number hidden before the model and not after it?
2. The output check found “promises money”. What does the customer get?
3. “Yes, you have a 10 € discount on 1042.” passes the checks. Why?
4. What should the log say about the second message?

Made by Ivo Gergov as part of Evo Tech's campaign for accessible programming education ·