Playbook for LLM agents building with Jev

TL;DR Jev answers narrow, typed questions about one state and returns choice/score/noul plus calibrated probabilities — it never generates text, does arithmetic, or picks its own next action. Decompose the user's job into atomic questions, filter the state down to what those questions need, ask them all in one POST /v1/systemone, and branch in code on the values and on confidence. Everything deterministic stays in your code.

Step 0 — Is Jev the right tool?

Work backwards from the behavior the user wants: what will the application show, select, change, or hand off? (The typesafe-ai agent skill and Claude Code plugin). Then place each decision in this table.

Do NOT use Jev for: generating text or code; math, counting, or reconstructing an exact number by interpolating between Score levels; date and time comparison; questions requiring several hops of indirection or double negatives; a huge state full of material the question does not need; anything a regular expression, parser, or if already answers exactly. Most of these are documented failure modes in Jev 1.13 jaggedness: known failure modes (nine in total); the last one — what a regular expression, parser, or if already answers — is not on that list but follows the how-to-build rule "Use code when you can" (How to build software with System One step 1).

One more thing to check before you write code: Jev's primary training language is English, and other languages including CJK are accepted but less accurate (Models, aliases, pricing, rate limits, context).

Step 1 — Decompose the job into questions

The docs call this "probably the most important concept" (How to build software with System One step 4).

1. List the decisions the workflow actually makes. Each one is either a code rule or a judgment. Keep control flow, deterministic rules, and side effects in code.
2. Split every judgment until each question tests exactly one property. If you cannot name the single property a question tests, it is not atomic yet. "Rate this startup pitch" becomes market size, technical feasibility, differentiation — weighted in code (Primitives: Choice, Score, Noul).
3. Pick a primitive per question using Choosing between Choice, Score, Noul: unordered options → Choice; a described spectrum → Score; a clean yes/no → Noul; a count → none, one Noul per item plus sum() in code; free text → none, propose candidates in code and let a Choice pick one.
4. Ask each decision one way. Structural invariants are not guaranteed: a Noul and a yes/no Choice on the same text returned 0.22 vs probabilities["yes"] = 0.01, and a question plus its negation summed to 1.19 (Jev 1.13 jaggedness: known failure modes #8). Do not implement "not X" by subtracting X, and do not carry a threshold tuned on a Noul over to a Choice. Enforce identities, totals, and mutual exclusion in code.
5. Add the speculative questions too. Questions run in parallel and adding them barely changes latency (Speculative fan-out) — but state each speculative premise explicitly, and ignore the answers on branches you did not take.

Step 2 — Shape the state

state is the single input every question in the request sees (State: what you send Jev).

• Shapes: a string, a JSON object, or an array of text values. Nothing else. Prefer an object with descriptive field names so questions can point at paths.
• Budgets: 64k tokens for state plus all questions; 32k tokens for state plus the single longest question. Staying under 64k does not guarantee you are under 32k (Models, aliases, pricing, rate limits, context).
• Filter first. Accuracy falls as the state grows with irrelevant detail, and a big state makes a wrong answer hard to localize (Jev 1.13 jaggedness: known failure modes #5). Retrieve and filter in code; where you cannot, use a Noul as a relevance filter (Cookbook: Classifying RAG passages).
• Pre-parse numbers and dates. Compute the number or the named bucket in code and put that in the state; Jev does better on semantic representations than numeric ones (Writing instructions and criteria that Jev reads correctly).
• Point at fields with backticked dot-and-index paths, backticks included: `ticket.messages[0].text`.
• Asymmetry to remember: extra questions are cheap, extra state is not.

Step 3 — Write instructions and criteria

Full guide: Writing instructions and criteria that Jev reads correctly. The rules that matter most:

• Literal reading. Jev answers the question you wrote, not the one you meant. When a wrong answer makes you say "but I meant…", that sentence belongs in instructions or in the criteria as a boundary case.
• Question ids are never sent to the model — write the complete question even when the id looks self-explanatory (HTTP API: POST /v1/systemone and GET /v1/models).
• Choice: give the full option list (up to 255 options per the launch blog; one cookbook reports ~240 as the practical working limit — Cookbook: Classification using confidence). Add an other/none of the above option whenever the list might not cover an input. For confusable options, escalate the description from a string to an object with what / not_for / examples.
• Score: criteria is an ordered array from low to high, at least 2 and up to 10 levels, each describing a concrete situation rather than a degree (Score questions). ["0","1","2"] with "rate 0 to 2" in the instructions scored 0.57 at confidence 0.35 where described levels scored 0.0 at 1.0. One dimension per Score.
• Noul: keep true meaning yes. A Noul whose true maps to "no" performs worse (Jev 1.13 jaggedness: known failure modes #7). Avoid double negatives — rewrite them positively.
• Align criteria with the instruction. Contradiction between the two is its own failure mode.

Step 4 — Call it

Get a key from the console: the quickstart points at https://console.typesafe.ai/settings/keys, the agent-skill page at https://console.typesafe.ai/keys — the sources disagree on the path, both are under console.typesafe.ai (console.typesafe.ai (console + playground)).

Environment variables read by both SDKs (TYPESAFE_* environment variables across SDKs):

Model ids: jev-latest and jev-preview both currently resolve to jev-1.13.0. Pin jev-1.13.0 if you have tuned thresholds; the response's model field reports which version answered (Models, aliases, pricing, rate limits, context).

curl

export TYPESAFE_API_KEY="..."   # from console.typesafe.ai

curl -X POST https://api.typesafe.ai/v1/systemone \
  -H "Authorization: Bearer $TYPESAFE_API_KEY" \
  -H "Content-Type: application/json" \
  -d @- <<'EOF'
{
  "state": {
    "ticket_message": "I was charged twice for order A-104. Please refund the duplicate.",
    "refund_policy": "Duplicate charges are eligible for a refund."
  },
  "model": "jev-latest",
  "questions": {
    "refund_requested": {
      "type": "noul",
      "instructions": "Does `ticket_message` request a refund?"
    },
    "department": {
      "type": "choice",
      "instructions": "Which team should handle `ticket_message`?",
      "criteria": {
        "billing": "Charges, invoices, refunds, subscriptions",
        "technical": "Bugs, outages, integrations",
        "other": "Anything else"
      }
    },
    "frustration": {
      "type": "score",
      "instructions": "How frustrated does the customer appear in `ticket_message`?",
      "criteria": [
        "Calm, just stating facts",
        "Frustrated but civil",
        "Very angry, strong language"
      ]
    }
  }
}
EOF

Python — typesafe-sdk 0.6.0, sync

# pip install typesafe-sdk    (import name: typesafe_sdk; requires Python >= 3.10)
from typesafe_sdk import Choice, Noul, Score, TypeSafeClient

STATE = {
    "ticket_message": "I was charged twice for order A-104. Please refund the duplicate.",
    "refund_policy": "Duplicate charges are eligible for a refund.",
}

QUESTIONS = {
    "refund_requested": Noul(instructions="Does `ticket_message` request a refund?"),
    "department": Choice(
        instructions="Which team should handle `ticket_message`?",
        criteria={
            "billing": "Charges, invoices, refunds, subscriptions",
            "technical": "Bugs, outages, integrations",
            "other": "Anything else",
        },
    ),
    "frustration": Score(
        instructions="How frustrated does the customer appear in `ticket_message`?",
        criteria=[
            "Calm, just stating facts",
            "Frustrated but civil",
            "Very angry, strong language",
        ],
    ),
}

with TypeSafeClient() as client:  # reads TYPESAFE_API_KEY; defaults to jev-latest
    response = client.system_one(state=STATE, questions=QUESTIONS, model="jev-1.13.0")

print(response.model, response.usage.input_tokens)
print(response.answers["refund_requested"].noul)
print(response.answers["department"].choice, response.answers["department"].confidence)
print(response.answers["frustration"].score, response.answers["frustration"].legend)

Python — async

import asyncio

from typesafe_sdk import AsyncTypeSafeClient, Noul


async def main() -> None:
    async with AsyncTypeSafeClient() as client:  # await client.aclose() if not using `async with`
        response = await client.system_one(
            state="Help! My payouts have been failing for 3 days.",
            questions={"is_urgent": Noul(instructions="Does this convey urgency?")},
        )
        print(response.answers["is_urgent"].noul)


asyncio.run(main())

Build one client per process and reuse it — do not construct a client per request (Python SDK: install, clients, system_one()).

TypeScript — @typesafe-ai/sdk 0.6.0

// npm install @typesafe-ai/sdk   (Node >= 20; server-side only unless you opt into
// dangerouslyAllowBrowser, which exposes your key)
import { APIError, choice, noul, score, TypeSafeClient } from "@typesafe-ai/sdk";

const client = new TypeSafeClient(); // reads TYPESAFE_API_KEY, TYPESAFE_BASE_URL, TYPESAFE_DEFAULT_MODEL

try {
  const { answers, usage, model } = await client.systemOne({
    state: {
      ticketMessage: "I was charged twice for order A-104. Please refund the duplicate.",
      refundPolicy: "Duplicate charges are eligible for a refund.",
    },
    model: "jev-1.13.0",
    questions: {
      refundRequested: noul("Does `ticketMessage` request a refund?"),
      department: choice("Which team should handle `ticketMessage`?", {
        billing: "Charges, invoices, refunds, subscriptions",
        technical: "Bugs, outages, integrations",
        other: "Anything else",
      }),
      frustration: score("How frustrated does the customer appear in `ticketMessage`?", [
        "Calm, just stating facts",
        "Frustrated but civil",
        "Very angry, strong language",
      ]),
    },
  });

  console.log(model, usage.input_tokens);
  console.log(answers.refundRequested.noul);
  console.log(answers.department.choice, answers.department.confidence);
  console.log(answers.frustration.score, answers.frustration.legend);
} catch (err) {
  if (err instanceof APIError) {
    console.error(`API error ${err.status} (request ${err.requestId ?? "unknown"}):`, err.body);
  } else {
    throw err;
  }
}

Answer types are inferred from the questions you passed, so answers.department.choice narrows to "billing" | "technical" | "other" (JavaScript/TypeScript SDK: install, client, choice/score/noul).

Step 5 — Use the answers

In Python, response.answers[...] is the mixed map and response.nouls / response.choices / response.scores are per-type views (Python SDK responses, answers, usage, models).

Confidence-gated routing. Check the floor before branching on the answer, then give each action its own threshold sized to the cost of being wrong (Confidence-gated routing):

These are examples from the sources, not defaults. Tune them on your own labelled data (Testing and evaluating a Jev workflow), and keep every question and threshold in one file so a human can review them (The typesafe-ai agent skill and Claude Code plugin).

Fan-out. Put every question the decision tree could need in one request; batching 13 questions about a 54k-character document was 12.2x cheaper and 10.0x faster than 13 single-question calls, with unchanged answers (Cookbook: Parallel questions, Speculative fan-out).

Composite scoring. Normalize each Score to 0–1, weight in code, and re-rank by changing a coefficient rather than re-running inference. Weighted sums are for compensating preferences; an "any serious violation" rule needs separate conditions (Composite scoring).

answers = response.answers
spam_risk = (
    0.45 * answers["requests_credentials"].noul
    + 0.30 * answers["sender_identity_mismatch"].noul
    + 0.25 * answers["unexpected_reward"].noul
)

Step 6 — Handle errors and limits

Both SDKs retry by default: 2 retries after the initial attempt, 500 ms initial backoff doubling to a 5,000 ms cap, 25% jitter, retryable statuses {408, 429, 500–599}; the JS SDK caps an honored server delay at 60,000 ms. Per-attempt HTTP timeout is 10 s (Python DEFAULT_TIMEOUT = 10.0, JS DEFAULT_TIMEOUT_MS = 10_000) (HTTP status codes, rate limits, retry semantics). Log the x-typesafe-request-id (response.request_id / err.requestId) on every failure.

Limits and cost (Models, aliases, pricing, rate limits, context): 250,000 tokens/second and 1,200 requests/minute, either breach returning 429; TypeSafe warns these "can change without notice," so treat 429 as normal rather than exceptional instead of hardcoding a client-side budget. Price is $42 per Btok / $0.042 per Mtok on input tokens only — output tokens are free. There is no uptime SLA (Legal: MCA, DPA, privacy, data retention).

Step 7 — Test before shipping

Do not ship on the strength of a few hand-checked examples.

• Consistency. Jev is not deterministic: 15 repeats of one claim moved a Noul from 0.43 to 0.53, across a 0.5 threshold (Cookbook: Self-consistency — nouls), and 2 of 8 Choice questions changed their top label (Cookbook: Self-consistency — choices).
• Held-out cases. Tune wording and thresholds on one set, validate on another (Writing instructions and criteria that Jev reads correctly).
• Jaggedness edge cases. Regression-test literal reading, negation, numbers, dates, and adversarial content (Jev 1.13 jaggedness: known failure modes).
• Full method, harness, and legal caveat: Testing and evaluating a Jev workflow.

Copy-paste checklist

• Every deterministic rule, sum, date comparison, and exact lookup is in code, not a question.
• Each question tests exactly one named property.
• Each decision is asked one way; invariants and totals are enforced in code.
• state is an object with descriptive keys, filtered to what the questions need, under 64k / 32k tokens.
• Questions point at state with backticked paths (`ticket.messages[0].text`).
• Choice has a none/other option; Score levels describe situations, 2–10 of them, one dimension; Noul true means yes.
• All questions for one state go in one request; a second request only when an answer is needed to build it.
• TYPESAFE_API_KEY set; key never shipped to a browser.
• model pinned to jev-1.13.0 if thresholds are tuned; response.model logged.
• confidence checked before branching on choice/score; Noul thresholded on noul itself.
• Questions and thresholds live in one reviewable file.
• 429/529 retried with backoff; 401/422 never retried; x-typesafe-request-id logged.
• usage.input_tokens metered against $0.042/Mtok.
• Repeatability, held-out accuracy, and adversarial inputs tested before rollout.
• TYPESAFE_LOG_LEVEL=debug is off in production (bodies are not redacted).

Common mistakes

Where to look next

Closest recipe by task type:

Related

• Consult guide: could Jev help this project? — not sure Jev fits the project at all? Run the consult procedure first (community tier)
• How to build software with System One — the seven-step design workflow this playbook operationalizes
• Patterns overview — the four architectural patterns
• The typesafe-ai agent skill and Claude Code plugin — TypeSafe's own instructions to coding agents
• Workflow evals: how TypeSafe measures Jev — evidence that the workflow shape beats a single prompt
• AI primer: why calibrated decision models — why the probabilities are calibrated
• Smart home assistant demo walkthrough — fan-out plus an LLM fallback in a running app

Sources

• wiki/concepts/system-one.md, wiki/concepts/primitives.md, wiki/concepts/state.md, wiki/concepts/confidence.md, wiki/concepts/how-to-build.md, wiki/concepts/jaggedness-jev-1-13.md, wiki/concepts/use-case-map.md, wiki/concepts/workflow-evals.md
• wiki/reference/http-api.md, wiki/reference/models-and-pricing.md, wiki/reference/rate-limits-and-errors.md, wiki/reference/python-sdk.md, wiki/reference/python-sdk-responses.md, wiki/reference/javascript-sdk.md, wiki/reference/environment-variables.md, wiki/reference/agent-skill.md, wiki/reference/legal-and-data.md
• wiki/patterns/overview.md, wiki/patterns/fan-out.md, wiki/patterns/confidence-routing.md, wiki/patterns/composite-scoring.md
• wiki/cookbooks/overview.md, wiki/cookbooks/parallel-questions.md, wiki/cookbooks/classification-using-confidence.md, wiki/cookbooks/consistency-choice.md, wiki/cookbooks/consistency-noul.md
• wiki/guides/quickstart.md, wiki/guides/choosing-a-primitive.md, wiki/guides/writing-instructions-and-criteria.md
• raw/docs/introduction__quickstart.md (https://docs.typesafe.ai/introduction/quickstart) — console key URL
• raw/docs/agent-skill.md (https://docs.typesafe.ai/agent-skill) — alternate console key URL