Build a workflow from small decisions

Draw the decisions before the calls

Sketch the application before choosing a pattern. For a return request, code knows the order date and return window. The message supplies less tidy evidence: whether the customer wants a replacement, whether an item is damaged, and which order they mean.

Mark where interpretation is required. Then ask whether those judgments share their evidence, whether they need different evidence, and what the application should do with an uncertain result. The sections below work through those choices.

Speculative fan-out

A customer says “A replacement would be great; otherwise I’d like a refund.” You can ask about the requested remedy and the details of each remedy using the same message and order.

Write each conditional question with its premise included: “If the customer wants a replacement, which item are they referring to?” The question cannot see the route answer beside it.

After the answers arrive, code selects a route and uses that route’s follow-ups. Questions for other routes may be irrelevant; their uncertainty should not block the chosen path.

This can save a repeated copy of the state and a network trip. It also asks questions that may go unused. Compare those costs for your request sizes instead of treating extra questions as free.

Composite scoring

You have six returns to review and only time for two. The order may depend on evidence quality, policy fit, account risk, and handling effort. Store each judgment separately so those priorities remain adjustable.

A single overall score is enough when you only need that one judgment. Use separate dimensions when you need to explain or change how the ranking is assembled.

Normalize the scales, choose weights, and sort in code. A weighted sum allows one strong factor to compensate for a weak one. Apply mandatory eligibility checks separately so a good score cannot cancel a disqualifying condition.

Confidence-gated cascade

Some returns can be routed from the first message. Others need an attachment, a follow-up, or a reviewer. A cascade gives those unresolved cases a next destination.

Keep the first stage small. Send cases that fail its tested acceptance rule to a stage with useful additional evidence or capability. Calling a larger model with the same missing facts may not resolve the problem.

Count the cost and delay of every stage reached. The lab assumes each stage resolves a chosen share of the cases it receives. The shares are inputs, not predictions.

Select, do not generate

The customer mentions two order numbers but wants to return only one. A parser can find both numbers; interpreting which one the customer means is a different task.

Collect candidate values in code, offer them as Choice options, and keep an option for no match. Once a candidate is selected, use the original value rather than asking a model to reproduce it.

Test candidate collection before testing selection. A missing candidate cannot win. Even a correct selection still needs authorization before you change the associated order.

import re
from typesafe_sdk import Choice, TypeSafeClient

message = "Returning the jacket from order R-20931. My other order R-20877 is fine."
candidates = re.findall(r"\bR-\d{5}\b", message)          # ["R-20931", "R-20877"]

criteria = dict.fromkeys(candidates)               # every candidate, no description
criteria["none"] = "No order number in the message is the one being returned"

with TypeSafeClient() as client:
    r = client.system_one(
        state={"message": message},
        questions={"which": Choice(
            instructions="Which order number does the customer want to return?",
            criteria=criteria,
        )},
    )

picked = r.choices["which"]
order_id = None if picked.choice == "none" else picked.choice  # copied, never retyped

For a large catalogue, you can narrow candidates in stages. Keep alternatives when an early branch is uncertain, and measure whether the extra paths improve your final selections.

Reuse the evidence within a request

Suppose a return message needs six independent checks. Six separate calls repeat the message six times. One call can include all six checks and pay the state cost once.

Split when a later request needs newly fetched evidence, a different state, or room beyond the current request budget. For latency, distinguish sequential calls from concurrent calls: parallel network requests can finish sooner, but still repeat the input.

Choosing a pattern

These compose. A real return-handling flow is a fan-out request whose branch answers feed a composite priority, gated by a cascade for the ambiguous cases, with order numbers pulled by selection. The next part builds one end to end.