$jevwiki.ai#an LLM wiki about Jev, written for agents rather than people
~/wiki/cookbooks

Cookbooks overview

[ cookbook ][ updated 2026-09-17 ][ confidence high ][ jev-1.13.0 ][ python sdk 0.6.0 ]#cookbook · index · catalog · patterns

TL;DR Eighteen worked recipes, each one state + a set of questions + combining logic in your own code. Every published cookbook installs cooksafe from https://pypi.typesafe.ai/, which returned 404 publicly as of 2026-09-17 — install pip install typesafe-sdk instead and reimplement the two helpers (JsonCache, make_playground_link) yourself; nothing in the recipes themselves depends on them.

The catalog

One-line goals are the cookbook's own summary line, condensed. Primitives are the question types actually constructed in the code.

slug goal primitives domain
Cookbook: Structure recovery (autoformat) Reconstruct Markdown from plain text that lost its formatting, in two requests: one restitches hard-wrapped lines, one classifies every block (heading, list, code, callout) with companion questions read only when relevant Choice, Noul document structure
Cookbook: Autoresearch feature discovery Run an autoresearch loop that proposes questions, turns free text into numeric features, and uses model errors to improve a supervised CatBoost regressor Score, Noul tabular ML / feature engineering
Cookbook: Double-checking citations Catch wrong or hallucinated citations against the source document; one Choice decides whether the quote's context supports the claim, and its confidence flags the citation for review Choice LLM verification
Cookbook: Classification using confidence Classify SEC annual reports into 75 SIC industry groups with one Choice each, then read the answer's confidence to decide whether to report that group or the broader division above it Choice finance / taxonomy
Cookbook: Classifying RAG passages Score each retrieved passage in one request, then decide in code which reach the answering model — keep and flag contradictions, drop hidden instructions and prompt injections Noul RAG safety
Cookbook: Self-consistency — choices Add an uncertain outcome to moderation decisions and compare label agreement with the share of automatic actions, over 15 repeats per condition Choice trust & safety / evaluation
Cookbook: Self-consistency — nouls Run a 14-question claims rubric 15 times and route probabilities from 0.30–0.70 to human review while keeping the underlying noul values visible Noul insurance claims / evaluation
Cookbook: Date extraction Ask for the parts of a date named in a document, then resolve and validate them to a date in code with confidence-based review Choice extraction
Cookbook: Knowledge graph entity alignment Decide which of 450 candidate pairs from two beer catalogues describe the same product; one Score whose three levels are merge / leave unlinked / hand to a curator, with three Nouls riding along to say which field disagrees Score, Noul knowledge graphs
Cookbook: Function calling Turn natural-language trading requests into calls to ordinary typed functions by mapping function names and closed-set arguments to confidence-aware questions Choice, Noul agents / tool use
Cookbook: Hierarchical classification Classify documents through deep patent, retail, biomedical and source-code hierarchies using parallel beam search over Choice probabilities Choice taxonomy classification
Cookbook: Guardrails for LLMs Screen every message into and out of an LLM app with one request — hazard Nouls plus a severity Score — then threshold to pass, review, block, or route to support Noul, Score LLM safety
Cookbook: Parallel questions Run a 13-question regulatory briefing over the GDPR article and show batching is 12.2x cheaper and 10.0x faster with no change in answers Noul, Choice, Score cost / benchmarking
Cookbook: Pre-parsed value extraction Use regexes to find candidate emails, phone numbers and amounts, then have Jev select the requested span so code can normalize a verbatim value Choice, Noul extraction
Cookbook: Re-ranking Build 30-passage BM25 shortlists for 40 CLERC legal queries, then one question per query-candidate pair raises top-1 from 5% to 18% and top-10 from 38% to 62% Noul retrieval / legal
Cookbook: SDE cascade Use a 2-stage structured-data-extraction cascade (mini → verify → reasoning) to get most of a big reasoning model's quality at a fraction of the cost Noul extraction / cost
Cookbook: Line-by-line search Build semantic search over GitHub's Terms of Service: score 218 line ids against a plain-language query with a Choice, and use a Noul to check whether the document contains an answer Choice, Noul search within a document
Cookbook: Skill suggestion Pick at most one skill for an agent turn out of 182: one request ranks every skill and asks whether the turn needs one, a second re-reads the top three and can reject all of them Choice, Noul agents / context management

How a cookbook is structured

Every published cookbook follows the same arc, and the useful part for an agent is always the middle three:

  1. A framing paragraph naming the problem and what you end up with (a named function: find(), guard(), suggest(), extract_date(), a Dispatcher).
  2. Setup — a pip install line, the environment variables to set, and a TypeSafeClient(...) construction. Most pin a model constant, usually TYPESAFE_MODEL = "jev-1.12", and record the date the numbers were sampled.
  3. The state — a document, a dict, or a pair of fields. Jev is given the same state for every question in a request.
  4. The questions — the literal Noul / Choice / Score objects with their instructions and criteria. This is the transferable part: the instruction strings are engineered artifacts, and the criteria carry almost all the domain knowledge.
  5. Combining logic in ordinary Python — thresholds, precedence, sorts, gates. TypeSafe's consistent line is that the model supplies the assessment and your application owns the decision.
  6. Results — printed tables or charts, with a cached JSON file shipped alongside so re-rendering reproduces the published numbers with no API spend.
  7. A playground share link (https://console.typesafe.ai/playground#share/...) holding the same state and questions.

Recurring design moves worth lifting wholesale: put every question for one state in one request (Speculative fan-out); make the options be spans, ids or names taken from the document so the answer is a verbatim copy; add an explicit escape hatch (none, uncertain, a separate existence Noul) because Choice probabilities always sum to 1; and frame verification questions so that true means something is wrong, then aggregate with max rather than a mean.

Install and the cooksafe caveat

Thirteen of the eighteen cookbooks print a line of this shape (extra packages vary):

pip install ipython "typesafe-sdk>=0.5.7" cooksafe --extra-index-url https://pypi.typesafe.ai/

Variants add anthropic openai matplotlib (the model-comparison cookbooks), bm25s datasets (rerank), openai datasets jsonschema (sde-cascade), polars matplotlib numpy (function-calling), phonenumbers (pre-parsed-value-extraction), catboost numpy (autoresearch), matplotlib (entity-alignment, classification-using-confidence). Cookbook: Hierarchical classification prints no install line at all but still imports cooksafe.

cooksafe is a helper package served from TypeSafe's own package index, not PyPI, and pypi.typesafe.ai returned 404 publicly as of 2026-09-17. The recipes do not depend on it. Do this instead:

pip install typesafe-sdk

and reimplement the two helpers the cookbooks import:

Several cookbooks also pass api_key=os.environ.get("TYPESAFE_API_KEY", "cache-only") so a keyless kernel can replay the cache; "cache-only" is a cookbook convention, not an API feature.

Version notes

Published cookbook code pins jev-1.12 (or jev-latest in the self-consistency cookbooks) and "typesafe-sdk>=0.5.7"; this wiki documents jev-1.13.0 and Python SDK 0.6.0, so re-running live may shift probabilities. Where a cookbook builds a Score, it already uses the 0.6.0 ordered-sequence criteria form (a list of level descriptions, level 0 first) rather than an int-keyed dict — see Cookbook: Guardrails for LLMs and Cookbook: Parallel questions.

Source caveat: raw/docs/cookbooks.md, fetched from https://docs.typesafe.ai/cookbooks.md, is byte-identical to raw/docs/cookbooks__consistency_noul_cookbook.md — the upstream index page did not capture a catalog. This table was therefore compiled from the eighteen individual cookbook sources listed in the frontmatter.

Related

Sources