Part 4 of 9 · Confidence and risk

Decide what happens next

A clear-looking answer still needs a policy. Use these experiments to explore the trade between automatic decisions, review effort, and mistakes.

Fictional expense claims and synthetic probabilities. Nothing on this page measures the model; it is about the policy around it.

Two numbers, two jobs

A Choice can favour “travel” while leaving a meaningful chance of “equipment.” The probabilities describe those alternatives. The API’s confidence field compresses the distribution into one measure of concentration.

Noul already has just two outcomes. Its one number is the probability of yes; subtract it from 1 to get the probability of no.

This invented response separates a category judgment from a policy check:

{
  "category": {
    "type": "choice",
    "choice": "travel",
    "probabilities": { "travel": 0.81, "meals": 0.03, "equipment": 0.01, "other": 0.15 },
    "confidence": 0.61   // illustrative; the API derives this from the shape above
  },
  "within_policy": {
    "type": "noul",
    "noul": 0.88         // a Noul has no confidence field
  }
}

A peaked distribution can still favour the wrong answer. Before using confidence as a routing signal, measure how often answers at similar confidence levels are correct on your own examples.

Read the shape first

The next exercise uses invented distributions and does not assign API confidence values. It shows what the probabilities reveal on their own.

Lab

Reading a distribution

Choose an evidence shape. Compare the leading answer with its alternatives, then read the practical implication. Every example is invented for this guide.

A distribution is a view of the model’s uncertainty. It does not reveal missing evidence or prove that the leading answer is correct.

Give uncertainty somewhere to go

An expense tool has more than two possible next steps. It can propose a category immediately, ask the submitter to confirm it, or send the whole claim to a reviewer. These actions carry different costs.

Set thresholds for the action being taken. Suggesting “travel” is easy to undo. Releasing a payment needs separate checks of amount, policy, and authorization.

Use the suggestionThe evidence clears your tested threshold.

Ask a narrow questionOne missing detail could settle the case.

Send for reviewThe evidence is conflicting or the consequence needs oversight.

In a branched workflow, only inspect the answers relevant to that branch. An uncertain meal-type answer should not block a claim that is clearly about a train journey.

Lab

Routing simulator

A synthetic pool of 300 expense claims, generated in this page. Move the two thresholds and watch the bands, the error rate among automatic approvals, and the reviewer load change. Nothing here came from a model.

Claims by assumed probability of correctness

Act at or above

0.85

Escalate below

0.50

Claims per day

Minutes per human touch

Each synthetic case has an assumed probability of correctness, and its outcome is sampled from that probability. The horizontal axis is that assumed probability, not API confidence. Random sampling means observed rates will vary.

Pick the threshold from costs, not from habit

A simplified cost model helps expose your assumptions. Let p be a validated probability of a correct decision, C_wrong the cost of an error, and C_review the cost of review.

Assume correct automatic decisions have no extra cost and review fixes every error. Then automation costs (1 − p) × C_wrong on average. Review costs C_review. The break-even rule is:

(1 − p) × C_wrong < C_review ⇔ p > 1 − C_review / C_wrong

At $3 per review and $40 per mistake, the break-even is 0.925. Treat it as a planning number: if reviewers also err, or outcomes differ in cost, fold those in.

Lab

Threshold from costs

Type your two costs. The break-even threshold comes from the formula above; the curve shows the realised cost per claim on the synthetic pool from the previous lab under a two-way policy (act or review) at every threshold. Illustrative.

Cost of one wrong automatic approval, $ (C_wrong)

Cost of one human review, $ (C_review)

Break-even p*

0.925

automate when p is above this

Lowest realised cost

$0.00

per claim, at threshold 0.90

Realised cost per claim vs. threshold

The curve is bumpy because the pool is small and the correct flags are random draws. On real data it is bumpy for the same reason, which is why you pick a threshold from a bucketed accuracy table rather than from a single lucky run. The formula assumes the reviewer is right and the model is calibrated; both are things you check, not things you assume.

Measure before you trust a threshold

Calibration is a statistical property. It holds on average over a population of judgments, and only the population you test on. Before a threshold gates anything that matters, build a labelled set from your own traffic and check how accuracy moves with the number you plan to gate on.

Label a few hundred real items. Real claims, real receipts, the messy ones included. Record the answer a careful person would give.
Run the exact questions you will ship. Same instructions, same criteria, same state shape. Changing a criterion changes the distributions, and with them every threshold.
Bucket by confidence, then by the winner's probability. Compare accuracy in each bucket. Keep whichever signal separates right from wrong more cleanly for your questions.
Read the thresholds off the table. The act threshold is the lowest bucket whose accuracy clears the cost-derived p*. Set the review boundary from the errors you can accept and the capacity of your review queue; a coin flip is not a meaningful baseline for every task.
Pin the model and re-run on change. The docs recommend pinning a versioned model ID once you have tuned thresholds, and moving to a new version on your own schedule. Re-run the table whenever criteria, state shape, or model change.

An eval sheet for the claims example might look like this (illustrative numbers):

Confidence bucket	Claims	Accuracy	Policy at C_review = $3, C_wrong = $40
0.90 to 1.00	212	97%	act clears p* = 0.925
0.75 to 0.90	131	89%	confirm below p*, review required
0.50 to 0.75	88	71%	confirm
below 0.50	69	48%	escalate review required

Part 7 shows how to build this table from a labelled set in a few lines of code (evaluation before shipping). Part 3 covers what the distributions themselves mean for each primitive (distribution explorer), and Part 6 turns bands into a full cascade to a slower model or a person (cascades).