Hold the credits before the agent runs, settle once after

Agent work is priced in credits, but a run's cost is unknown until it ends. A hold before the run and one settlement after it keep the balance honest.

Run request Estimator Credit hold Agent run Rater Settlement Ledger Refusal worst case rate card append-only capture, release not enough balance
A run is priced at its worst case and held, refused up front if the balance is short, then settled once into the ledger.
On this page
  1. Two holes in metering afterwards
  2. What each stop on the line does
  3. One ledger table, three statements
  4. Tarnbrook Collect, with numbers
  5. Customers with credits who still get refused

A hotel authorizes more than the room costs and captures the real bill at checkout. Agent runs sold by the credit need the same move, because nobody knows what a run costs until it ends.

Credits have become a common unit for selling agent work, and a run’s price depends on how many accounts the agent gets through, how much it writes, and how many emails it decides to send. So most first versions meter afterwards: let the run finish, add up the usage, subtract it from the balance.

Two holes in metering afterwards

The first hole is the empty balance. If the check is “balance above zero, go”, a customer with 40 credits can start a run that costs 700. Now you either eat 660 credits, send a surprise overage, or kill the run halfway. Two runs started a second apart both pass the same check against the same balance, which makes it worse.

The second hole is the retry. The worker finishes, calls the billing service to deduct, and the call times out. The worker can’t know whether the deduction landed, so it retries, and a deduction written as “subtract 682” happily subtracts 682 twice.

The hotel move closes the first hole. Hold the worst case before the run, capture the actual after it, and release the difference. Key the capture to the run so it can only be written once, and the second hole closes too.

What each stop on the line does

The estimator turns a run request into a ceiling. It has to be a real ceiling, which means the agent needs hard caps to multiply: a maximum number of accounts, a cap on output tokens per account, at most one email each. Without caps there’s no worst case to hold.

The hold checks the available balance and reserves the ceiling in one transaction. If the balance can’t cover it, the run is refused there, before any tokens are spent. The hold also pins the rate card version, so a price change mid-run doesn’t reprice work already promised.

The agent then runs and emits usage events: account researched, email sent, plus the token counts that come back with every model call. The rater multiplies those events by the pinned rate card. Settlement writes the capture and the release together, keyed by run id, and everything lands in a ledger that is only ever appended to.

One ledger table, three statements

This is Postgres, and it leans on two old features: a row lock and a unique constraint.

create table credit_account (
  account_id bigint primary key,
  name       text not null
);

create table credit_ledger (
  entry_id          bigint generated always as identity primary key,
  account_id        bigint not null references credit_account,
  entry_type        text   not null
    check (entry_type in ('grant', 'hold', 'capture', 'release', 'expire')),
  credits           bigint not null check (credits >= 0),
  run_id            uuid,
  rate_card_version int,
  created_at        timestamptz not null default now(),
  check (entry_type in ('grant', 'expire') or run_id is not null),
  check (entry_type in ('grant', 'expire') or rate_card_version is not null),
  unique (run_id, entry_type)   -- one hold, one capture, one release per run
);

create index credit_ledger_account_idx on credit_ledger (account_id);

-- The application role may insert and select, nothing else:
-- revoke update, delete, truncate on credit_ledger from app_writer;

-- Every closed run nets to its capture (hold minus release), so this is
-- granted minus expired minus captured minus open holds.
create view credit_balance as
select a.account_id,
       coalesce(sum(case l.entry_type
                      when 'grant'   then  l.credits
                      when 'release' then  l.credits
                      when 'hold'    then -l.credits
                      when 'expire'  then -l.credits
                      else 0
                    end), 0) as available,
       coalesce(sum(l.credits) filter (where l.entry_type = 'capture'), 0) as captured
from credit_account a
left join credit_ledger l on l.account_id = a.account_id
group by a.account_id;

-- 1. Place a hold. $1 account, $2 run id, $3 worst case, $4 rate card version.
--    Run at READ COMMITTED so the insert sees holds committed while it waited.
begin;
select 1 from credit_account where account_id = $1 for update;

insert into credit_ledger (account_id, entry_type, credits, run_id, rate_card_version)
select $1::bigint, 'hold', $3::bigint, $2::uuid, $4::int
where (select available from credit_balance where account_id = $1::bigint) >= $3::bigint
on conflict (run_id, entry_type) do nothing;
commit;
-- Zero rows inserted: the hold already exists (a retry) or the run is refused.

-- 2. Settle. $1 run id, $2 rated credits. The release row closes the run,
--    so whoever writes it first wins and every later attempt inserts nothing.
with h as (
  select account_id, credits, rate_card_version
  from credit_ledger
  where run_id = $1::uuid and entry_type = 'hold'
),
closed as (
  insert into credit_ledger (account_id, entry_type, credits, run_id, rate_card_version)
  select account_id, 'release', credits - least($2::bigint, credits),
         $1::uuid, rate_card_version
  from h
  on conflict (run_id, entry_type) do nothing
  returning run_id
)
insert into credit_ledger (account_id, entry_type, credits, run_id, rate_card_version)
select h.account_id, 'capture', least($2::bigint, h.credits),
       $1::uuid, h.rate_card_version
from h
join closed on true;

-- 3. Sweep holds older than the run timeout that nobody closed.
insert into credit_ledger (account_id, entry_type, credits, run_id, rate_card_version)
select h.account_id, 'release', h.credits, h.run_id, h.rate_card_version
from credit_ledger h
where h.entry_type = 'hold'
  and h.created_at < now() - interval '30 minutes'
  and not exists (
    select 1 from credit_ledger r
    where r.run_id = h.run_id and r.entry_type = 'release'
  )
on conflict (run_id, entry_type) do nothing;

A few choices in there are deliberate. The lock is on the account row, so two holds for one customer queue up while other customers aren’t touched. The capture is capped at the hold with least, because the hold is the promise made to the customer; if the agent overshoots, that’s my bug and my cost. And the release row is written even when it’s zero, since its existence is what marks a run as closed.

Purchased packs and monthly allowances arrive as grant rows from whatever turns a signed order into billing, and unused ones leave as expire rows.

Tarnbrook Collect, with numbers

Tarnbrook Collect is a product I made up for this post, figures included. It chases overdue invoices, and it sells an agent that researches each overdue account and sends a reminder where one is warranted. Rate card v3 charges 4 credits per account researched, 1 credit per 1,000 output tokens, and 10 credits per email sent. A credit is $0.02.

A customer with 1,200 credits asks for a run over 60 accounts. Output is capped at 3,000 tokens per account, so the ceiling is 60 x (4 + 3 + 10) = 1,020. The hold goes in and 180 credits stay available.

While that run is working, someone at the same customer requests a second one for 25 accounts. Its ceiling is 425 against 180 available, so it’s refused on the spot with a message that says why. The first run’s real cost isn’t known yet, and the ledger won’t promise credits it might not have.

The first run finishes: 60 accounts researched (240), 62,000 output tokens (62), 38 emails sent (380). The rater says 682.

Entry Credits Available after
grant 1,200 1,200
hold, run A, card v3 1,020 180
capture, run A 682 180
release, run A 338 518

The settle call times out on the worker’s side, though it committed. The worker retries with the same run id, the release insert hits the unique constraint, and nothing is written. The balance stays at 518, and a 25-account run requested now would get its hold.

Rating after the run works the same way as usage billing from events, pinned prices included. The new part is the hold that comes before it.

Customers with credits who still get refused

Holds are pessimistic. That customer had 1,200 credits and could use only 180 of them while the run was in flight, on a run that ended up needing 682. If the estimator is lazy, customers with healthy balances get refused, and they’ll notice. I’d track the ratio of captured to held per agent and tighten the caps until it’s reasonable, or split big runs into batches that each hold a smaller amount.

The sweep has a sharper edge. The statement above releases the whole hold, which is right when a run died before doing anything. If it crashed after sending twenty emails, a full release gives that work away. I’d have the sweeper rate whatever usage events exist and call the settle statement with that figure, keeping the plain release for runs with no events.

The balance view scans the account’s whole history, which is fine for a long while. When it isn’t, add a snapshot row per account per month and sum from there. The ledger stays the source of truth either way.