Decide what happens next

Two numbers, two jobs

A Choice can favour “travel” while leaving a meaningful chance of “equipment.” The probabilities describe those alternatives. The API’s confidence field compresses the distribution into one measure of concentration.

Noul already has just two outcomes. Its one number is the probability of yes; subtract it from 1 to get the probability of no.

This invented response separates a category judgment from a policy check:

{
  "category": {
    "type": "choice",
    "choice": "travel",
    "probabilities": { "travel": 0.81, "meals": 0.03, "equipment": 0.01, "other": 0.15 },
    "confidence": 0.7466666667  // (0.81 - 1/4) / (1 - 1/4)
  },
  "within_policy": {
    "type": "noul",
    "noul": 0.88         // a Noul has no confidence field
  }
}

Interpret confidence using the selected implementation’s contract. TypeSafe now publishes the Jev formulas: Choice compares its winning probability with an even split; Score accounts for distance from its most likely level. Laya documents an entropy-based statistic; the same field name does not make cutoffs interchangeable. Choose and validate the signal again when moving providers. See the compatibility notes. A peaked distribution can still favour a wrong answer, including when needed evidence is missing. The frame’s coverage and observed action errors need their own checks.

Compute the documented Jev signals

For Choice with n options, confidence is (p_max − 1/n) / (1 − 1/n). With four options and a winning probability of 0.81, it is about 0.747. The remaining probabilities do not affect this statistic.

For Score, find the most likely level m. Compute spread = Σ p_i × |i − m| and the uniform baseline U = Σ |i − (n−1)/2| / n. Confidence is max(0, 1 − spread/U). Probability far from the mode costs more than probability on a neighbouring level.

A Noul has no separate confidence field. An optional derived measure is |2p − 1|; retain the original yes probability so direction is preserved. Box confidence has its own contract. Source: TypeSafe’s published formulas, checked October 15, 2026.

Give uncertainty somewhere to go

An expense tool has more than two possible next steps. It can propose a category immediately, ask the submitter to confirm it, or send the whole claim to a reviewer. These actions carry different costs.

Set thresholds for the action being taken. Suggesting “travel” is easy to undo. Releasing a payment needs separate checks of amount, policy, and authorization.

In a branched workflow, only inspect the answers relevant to that branch. An uncertain meal-type answer should not block a claim that is clearly about a train journey.

Pick the threshold from costs, not from habit

A simplified cost model helps expose your assumptions. Let p be a validated probability of a correct decision, C_wrong the cost of an error, and C_review the cost of review.

Assume correct automatic decisions have no extra cost and review fixes every error. Then automation costs (1 − p) × C_wrong on average. Review costs C_review. The break-even rule is:

At $3 per review and $40 per mistake, the break-even is 0.925. Treat it as a planning number: if reviewers also err, or outcomes differ in cost, fold those in.

Validate the rule on the region it accepts

Calibration compares predictions with observed frequencies on a defined population. A routing rule needs further evidence: how often its accepted actions are correct, which classes it misses, and how much traffic goes to review or receives no answer.

1. Separate tuning and final reporting. Build representative labelled sets before trying thresholds. Include missing evidence, minority classes, and difficult messages. Sample size depends on the claim you need to support.
2. Run the request you plan to ship. Keep its state construction, criteria, option order, batch form, and resolved model recorded. Retain failures as failures.
3. Select a signal on tuning data. Compare returned confidence and relevant probabilities. Check per-action precision and recall, not just the mean confidence or overall accuracy.
4. Inspect the whole accepted region. Use cumulative counts above the proposed cutoff, class coverage, and uncertainty estimates. A small perfect bucket can be misleading. The cost-derived break-even concerns correctness probability, not raw confidence.
5. Fix the policy, then test it. Check it once on untouched data. Re-evaluate after changes to the request, model, traffic, or action costs.

The denominator lab makes the counts visible. The Jev studies show why repeated stable labels can still produce action flips near a cutoff. Use a cascade when an unresolved case needs more evidence or a different reviewer.