Part 7 of 9 · Build guide

Try one useful integration

Build a small issue-triage assistant: read a report, propose labels, and defer unclear cases. Edit a request, inspect an example response, and practise evaluating the result before connecting it to real issues.

The project and its issues are fictional; the builder runs locally and sends nothing.

Get a key, pick an SDK

Keep the TypeSafe key in your server environment. Choose an SDK or send HTTP requests directly. The snippets below show the starting point; follow the linked SDK reference for version-specific configuration.

pip install typesafe-sdk        # or: uv add typesafe-sdk
export TYPESAFE_API_KEY=sk-...   # from the TypeSafe console

npm install @typesafe-ai/sdk     # needs Node 20+
export TYPESAFE_API_KEY=sk-...

export TYPESAFE_API_KEY=sk-...
# POST https://api.typesafe.ai/v1/systemone with Authorization: Bearer $TYPESAFE_API_KEY

The optional TypeSafe agent skill supplies API guidance to a coding assistant. You can use this tutorial without installing it.

The request, annotated

A fictional queue library receives a report that its consumer stops after a broker restart. We want to propose an issue type, identify missing information, and assess the described impact if it is a defect. Keep the first version modest: suggest labels for a maintainer to review. Sending comments or paging an engineer adds consequences that need their own acceptance rules and duplicate-action protection.

{
  "state": {
    "repo": "meridian/relay-queue",
    "issue": {
      "title": "Consumer stops pulling after broker restart",
      "body": "After we restart the broker, the consumer logs reconnected once and then idles forever. Jobs pile up until the process is restarted. Happens on 3.2.0, worked on 3.1.x. No stack trace, CPU flat. Config attached.",
      "author_association": "CONTRIBUTOR",
      "comments": 0
    }
  },
  "model": "jev-latest",
  "questions": {
    "kind": {
      "type": "choice",
      "instructions": "What kind of issue is `issue`, judged from its title and body?",
      "criteria": {
        "bug": "Behaviour that used to work or is documented does not behave as described",
        "feature": "A request for behaviour the project does not claim to have",
        "question": "The author is asking how to do something, not reporting a fault",
        "docs": "The code is fine but the documentation is wrong or missing",
        "none_of_these": "Spam, empty, or off-topic for this repository"
      }
    },
    "needs_info": {
      "type": "noul",
      "instructions": "Would a maintainer have to ask the author for more before they could start work on `issue`?",
      "criteria": {
        "true": "Something needed to act is missing: a version, a reproduction, logs, or configuration",
        "false": "The report contains enough to reproduce the problem or to decide what to do"
      }
    },
    "severity": {
      "type": "score",
      "instructions": "If `issue` describes a defect, how bad is its impact on a user of the library?",
      "criteria": [
        "Cosmetic or a minor annoyance",
        "A feature is degraded but a workaround exists",
        "A feature is unusable and no workaround is described",
        "Data loss, corruption, or a security exposure"
      ]
    }
  }
}

The state is a JSON object, not a blob. Named fields let the instructions point at parts of it with backticked paths such as issue. Facts code already knows, like the author's association, ride along as fields rather than being asked.
Ids exist for your code, not the model. The docs are explicit that the id never reaches the model, so the meaning has to be complete inside instructions and criteria. A key like kind tells the model nothing.
Each type has its own criteria shape. Choice takes a map of option to description, with null allowed when an option needs none. Score takes an ordered array of at least two level descriptions. Noul's criteria are optional and describe what yes and no mean.
There is always somewhere for the mass to go. none_of_these gives the Choice an exit. Without it, an off-topic issue is forced into the closest wrong label with an honest-looking probability.
The severity question is speculative. It only matters when kind comes back as a bug, but it is asked in the same request because the state is already there. See fan-out.

Lab

Request builder

Edit the state and the questions; the JSON body, curl command, Python, and TypeScript update as you type. Prefilled with the triage example. Nothing is sent anywhere.

Input format

State to evaluate

Questions

Token counts are estimated at four characters per token and are only there to keep the documented budgets in view: 64k tokens per request, 32k for the state plus the longest question. The generated SDK code uses the documented class and method names; the TypeScript sample passes raw question objects for Score and for Noul with criteria, next to the documented choice() and noul() helpers.

Lab

Response reader

An illustrative response to the triage request. Click a field to see what it means and the line of code that reads it.

Every answer carries its type. Choice and Score answers carry confidence, derived from the distribution; Noul answers carry only the probability. Output tokens are billed at zero under the documented pricing, so the number that matters for cost is input_tokens.

Errors and operations

Treat a failed request separately from an uncertain answer. A timeout gives you no judgment at all. Keep the issue in a review queue so the failure remains visible.

Status	Documented meaning	What your code should do
401	Missing or invalid API key	Do not retry. Fix the `Authorization` header or the environment variable, and alert: this is a deploy problem, not a traffic problem.
422	Request failed validation; the body names the offending field	Do not retry. Log the body, fix the question shape. Common causes: a Score with one level, a Choice with no criteria map, a missing `type`.
429	Rate limit exceeded	Retry with exponential backoff and honor `retry-after` when present. Both SDKs do this by default.
529	Service temporarily overloaded	Same as 429. Cap the retries and fall through to your cascade's next stage rather than blocking the request path.

Use bounded retries for transient failures and honour a server retry delay when provided. Set a timeout that fits your application. Check the SDK documentation for the retry controls in the version you use.

Retries can repeat evaluation. Protect downstream actions with the issue ID and a processing version so a retry cannot post the same comment or apply the same operation twice.

Watch both request rate and token rate. Batching reduces repeated input and request count, but larger batches still consume tokens. Consult current limits when sizing traffic.

from typesafe_sdk import RetryPolicy, TypeSafeAPIError, TypeSafeClient

policy = RetryPolicy(max_retries=2, timeout=3.0)   # short: a user is waiting
client = TypeSafeClient(retry=policy)

def triage(issue):
    try:
        r = client.system_one(state=build_state(issue), questions=QUESTIONS, model=MODEL)
    except TypeSafeAPIError as err:
        if err.status in (401, 422):
            raise                      # our bug: surface it
        return enqueue_for_manual_triage(issue)   # 429 or 529 after retries: degrade
    kind = r.choices["kind"]
    if kind.confidence < 0.55:
        return enqueue_for_manual_triage(issue)
    apply_label(issue, kind.choice)
    if r.nouls["needs_info"].noul > 0.7:
        post_needs_info_comment(issue)
    if kind.choice == "bug" and r.scores["severity"].score >= 2.5:
        page_on_call(issue)

Evaluate before you ship

Run the proposed workflow on historical issues before it changes real ones. Track wrong labels and how much work reaches maintainers. Reserve a separate labelled set for checking the threshold you choose, so the same examples do not both tune and validate the rule.

Collect a few hundred past issues with the labels maintainers actually applied. Include the awkward ones.
Run the exact production request over each. Store state, questions, the response, and the model field that answered.
Bucket by confidence. For each bucket, compute accuracy and coverage. Pick the lowest band whose cumulative accuracy above it meets your bar.
Repeat when you change a criterion, and when the model alias moves.

Lab

Eval sheet

Forty synthetic labelled issues with the model's kind answer and confidence. Change the band width and the target accuracy to see where an "act" threshold would land. The sample is generated in the page and is illustrative only.

Band width

Target accuracy (%)

Confidence band	n	Accuracy	Acc. above	Coverage above

Labelled sample

Issue	Label	Answer	Conf.

"Acc. above" is the accuracy of everything at or above the band's lower edge; "coverage above" is how much of the sample that is. The suggested threshold is the lowest edge whose accuracy above meets the target. With forty items the bands are noisy, which is the point: a real eval needs hundreds. Part 4 covers what to measure.

Connect it to real traffic carefully

Keep the key behind your server endpoint. The browser should receive only the information it needs to display the result.
Record the evaluated version. An alias such as jev-latest can move. If you tune thresholds for one version, pin it and evaluate a replacement before changing it.
Collect useful feedback. Store the question version, output, and maintainer correction. Retain only the source text needed for evaluation, with suitable access controls and retention.
Keep a fallback path. Service errors, missing evidence, and unresolved cases should remain visible to maintainers.

For environment options, SDK behaviour, and migration details, use the official SDK reference. Those details change more often than the workflow above.