refund_order is a sensible name for a tool. The returns desk doesn’t have one called that, and an 8B model called it anyway.
{"name": "refund_order", "arguments": "{\"order_id\": \"FH-20418\", \"reason\": \"arrived wilted\"}"}
The real tool is issue_refund. It wants a line item and an amount, and it shouldn’t have been on offer in the first place, because nobody had checked the return window yet.
I find this kind of failure encouraging. The model understood the customer fine. What it got wrong is mechanical, and mechanical errors can be caught by code that costs almost nothing to run. So I treat a small model as a proposer. It suggests a call, and the call has to get past a shortlist, a grammar and a validator before anything executes.
The rest of this post builds that setup one guard at a time. The desk belongs to Fernhollow Plants, a mail-order nursery that doesn’t exist. The call above and every number below are as invented as the company. It runs an 8-billion-parameter open-weight model, the Llama 3 8B size class, on one GPU with seven tools: look up order, check return window, create label, issue refund and so on.
290 of 500, filed by how they failed
Before any guard goes in there’s a fixed task set: 500 recorded returns conversations, each with a grader that checks the end state. The label has to be for the right order, the refund on the right line for the right amount.
The bare model, with all seven tools in the prompt, completes 290. That’s 58%.
The 210 failures are more useful than the score, so each gets filed under the first thing that went wrong. 74 called a tool that wasn’t legal at that step. 19 named a tool that doesn’t exist. 47 sent arguments that didn’t parse, and 58 sent a value the tool rejects. The last 12 made a valid call that did the wrong thing.
So 198 of 210 are mechanical. The guards go in one at a time with a full rerun after each, the same one-change discipline I argued for with coding agents. Add two together and you can’t say which one paid.
356: the model only sees tools it could legally call
A return is a small state machine: no order yet, order loaded, window open. In each state only two to four of the seven tools make sense, so those are the only ones the prompt describes. Until the window check passes, issue_refund isn’t in the prompt and can’t be proposed. That’s the ordering the path check in grading the agent’s path catches after the fact. Here the shortlist enforces it before the call exists.
It’s the cheapest guard of the four, a dictionary. 356 of 500.
That’s a gain of 66 against 93 tool-choice failures in the tally. A task that gets past one failure is free to hit the next.
409: arguments decoded against a grammar
Constrained decoding masks every token that would break a schema. Outlines does it for open models, and most local inference servers accept a JSON schema or a grammar directly. I build one schema per state: a union with a branch per shortlisted tool, the name pinned as a constant, the arguments as that tool’s own schema. Enums come along, so reason can only be one of the five values the refund tool takes.
409 of 500.
The grammar has two blind spots. A call cut off at the token limit is still broken JSON. And it knows nothing about data, so a line_id can match the pattern perfectly and not be on this order.
447: a validator, and one chance to fix the call
So a validator runs after the model even with the grammar in place. JSON Schema first, then cross-field rules written as plain functions: the line belongs to the order, the refund doesn’t exceed what was paid for that line.
A rejected call doesn’t run. The validator’s message goes back to the model as the next turn, once. The message follows the format from a wrong tool call is usually a schema bug: the field, what arrived, and what’s allowed.
447 of 500.
479: the third attempt belongs to a larger model
If the repair is rejected too, I stop asking the small model. A second repair turn mostly returns the same mistake in different words. The call goes to a larger hosted model, with the same shortlist and the same validator in front of it, because bigger doesn’t mean trusted. If that call is rejected as well, a person gets the conversation.
479 of 500, or 95.8%. And 41 tasks, 8.2%, had a call escalated.
guarded_call(), start to finish
import json
from collections import Counter
from jsonschema import Draft202012Validator
LEGAL = { # state -> the only tools the model is shown
"no_order": {"look_up_order", "hand_to_human"},
"order_loaded": {"check_return_window", "add_order_note", "hand_to_human"},
"window_open": {"create_return_label", "issue_refund", "send_replacement", "hand_to_human"},
}
failures = Counter() # the grader adds "valid_but_wrong" after each run
class Rejected(Exception):
def __init__(self, failure_class, detail):
super().__init__(detail)
self.failure_class, self.detail = failure_class, detail
def refund_fits_line(args, ctx): # cross-field rule: a message, or None if fine
line = ctx["order"]["lines"].get(args["line_id"])
if line is None:
return f"line_id {args['line_id']} is not on this order. Lines: {sorted(ctx['order']['lines'])}."
if args["amount_cents"] > line["paid_cents"]:
return f"amount_cents is {args['amount_cents']} but only {line['paid_cents']} was paid for that line."
TOOLS = { # six more entries shaped like this one
"issue_refund": {
"description": "Refund one line of the loaded order.",
"schema": {
"type": "object",
"properties": {
"line_id": {"type": "string", "pattern": "^L[0-9]+$"},
"amount_cents": {"type": "integer", "minimum": 1},
"reason": {"enum": ["dead_on_arrival", "damaged", "wrong_item", "late", "other"]},
},
"required": ["line_id", "amount_cents", "reason"],
"additionalProperties": False,
},
"rules": [refund_fits_line],
},
}
def check(call, state, ctx):
name, allowed = call.get("name"), sorted(LEGAL[state])
if name not in TOOLS:
raise Rejected("unknown_tool", f"There is no tool named {name!r}. Choose from {allowed}.")
if name not in LEGAL[state]:
raise Rejected("illegal_in_state", f"{name} can't be called yet. Choose from {allowed}.")
try:
args = json.loads(call.get("arguments") or "{}")
except json.JSONDecodeError as e:
raise Rejected("unparseable_arguments", f"Arguments are not valid JSON: {e.msg} at character {e.pos}.")
for err in Draft202012Validator(TOOLS[name]["schema"]).iter_errors(args):
field = "/".join(str(p) for p in err.path) or "arguments"
raise Rejected("invalid_value", f"{field}: {err.message}")
for rule in TOOLS[name].get("rules", []):
problem = rule(args, ctx)
if problem:
raise Rejected("invalid_value", problem)
return name, args
def guarded_call(messages, state, ctx, small, large, execute):
shortlist = [{"name": n, "description": TOOLS[n]["description"], "parameters": TOOLS[n]["schema"]}
for n in sorted(LEGAL[state])]
for model, attempt in ((small, "first"), (small, "repair"), (large, "escalated")):
if attempt == "escalated":
failures["escalated"] += 1
call = model(messages, shortlist) or {}
try:
name, args = check(call, state, ctx)
except Rejected as r:
failures[r.failure_class] += 1
messages = messages + [
{"role": "assistant", "content": json.dumps(call)},
{"role": "user", "content": f"That call was rejected and did not run. {r.detail} Send one corrected call."},
]
continue
return execute(name, args), attempt
return None, "hand_to_human" # three rejections: a person takes over
small and large are callables that take the messages and the shortlist and return {"name": ..., "arguments": "<json string>"}, the shape function-calling APIs settled on in 2023. The grammar lives inside small.
The 21 calls that passed every check
Twelve are the small model’s, the same twelve from the first tally, and no guard ever questioned them. Nine are the larger model’s: its call passed the validator and was still wrong.
The typical one goes like this. A customer bought a fern and a ficus, the fern arrived dead, and the refund went to the ficus. The tool is legal in that state, the line id is real and on the order, and the amount is within what was paid.
These don’t get another guard. They become eval cases tagged valid_but_wrong, and the fixes are a different kind of work: a tool result that returns line descriptions next to line ids, a sharper tool description, maybe a confirmation step before money moves. I’d much rather do that work on 21 tasks than on 210.
The scoreboard, and the column I’d manage
Copy this and fill it in from your own task set.
| Guard added | Tasks passed (of N) | Calls escalated |
|---|---|---|
| Bare model, all tools | ||
| + state shortlist | ||
| + constrained arguments | ||
| + validator, one repair turn | ||
| + escalation after second rejection |
The pass rate will get the attention. I’d manage the right-hand column, because it says whether the small model still pays. Every escalated call costs the larger model’s price plus two wasted small-model turns and the wait. In the triage post the escalation signal was a confidence score. A tool call has no score worth trusting, so a rejection is the signal, and the rate still needs a ceiling. Set it before you build. Fernhollow’s is 15% of tasks, so 8.2% leaves room.
When a new tool or a policy change pushes escalations toward that ceiling, I wouldn’t reach for the bigger model first. The failure counter keeps running in production, and it already says which class moved.