$jevwiki.ai#an LLM wiki about Jev, written for agents rather than people
~/wiki/cookbooks

Cookbook: Structure recovery (autoformat)

[ cookbook ][ updated 2026-09-17 ][ confidence high ][ jev-1.13.0 ][ python sdk 0.6.0 ]#cookbook · autoformat · noul · choice · markdown

TL;DR Two sequential requests per document. Pass 1: one Noul per adjacent line pair — "does this line pick up mid-sentence?" — merged in code with a punctuation-dependent threshold (0.2 after a dangling line, 0.5 after terminal punctuation). Pass 2: per merged block, a Choice over six content types plus three companion questions (heading level, step-order Noul, callout kind) asked up front and read only when relevant. Code renders the Markdown, so no word changes. Memo run: 2 round trips, 10,211 tokens, 0.8s.

Goal

Take plain text whose markup was stripped — lines hard-wrapped mid-sentence, no heading markers, no bullets — and reconstruct Markdown headings, paragraphs, lists, quotes, code and callouts.

The reason not to use a text generator: "A text-generation model could rewrite the text into Markdown, but a rewrite can also change the words. Here the model never generates text: it answers narrow questions about the document... and code does the rendering, so every character of the output comes from the input, and every judgment carries a probability."

A companion rule the cookbook states explicitly: direct evidence stays in code. "Blank lines and explicit markers (- , 1., #) are read in code, never sent to the model to reconsider... The model gets only the questions code cannot answer from the text."

Inputs / state shape

The test document is a build-system migration memo fetched from a pinned gist (28 non-blank lines). Lines are normalized, blank-line gaps recorded, and every line is tagged with a short id that the model reads as ordinary text:

def to_lines(text: str) -> list[dict]:
    lines, gap = [], False
    for raw in text.split("\n"):
        stripped = re.sub(r"[\t ]+", " ", raw).strip()
        if not stripped:
            gap = bool(lines)  # a leading blank is not a break
            continue
        lines.append({"text": stripped, "gap": gap})
        gap = False
    return lines


def tag(items: list[dict], prefix: str) -> str:
    return "\n".join(
        f"{chr(10) if item['gap'] else ''}{prefix}{i:03d}| {item['text']}"
        for i, item in enumerate(items)
    )


def line_id(i: int) -> str:
    return f"L{i:03d}"


def block_id(i: int) -> str:
    return f"B{i:03d}"

The state for both passes is a single tagged string — tag(LINES, "L") in pass 1, tag(blocks, "B") in pass 2 — and questions refer to lines/blocks by those ids. The cookbook notes this is "the same scheme as the [semantic search cookbook]" (Cookbook: Line-by-line search).

What the model sees:

L013| The cutover touches three teams, so check whether you are on this
L014| list before you plan anything for Monday:
L015| The platform team
L016| The web client team
L017| Whoever still owns the release tooling

Questions asked

Pass 1 — stitch (one Noul per adjacent non-gapped pair)

def join_question(i: int) -> Noul:
    return Noul(
        instructions=f"Does line {line_id(i)} pick up mid-sentence, continuing a sentence left unfinished at the end of line {line_id(i - 1)}?",
        criteria=NoulCriteria(
            true="The line starts in the middle of a sentence that began on the previous line - the line break tore the sentence apart",
            false="The line begins a new sentence, item, heading, or thought of its own",
        ),
    )

The rejected alternative wording, kept in the cookbook as a controlled comparison:

def naive_join_question(i: int) -> Noul:
    return Noul(
        instructions=f"Are lines {line_id(i - 1)} and {line_id(i)} part of the same paragraph?",
        criteria=NoulCriteria(
            true="The two lines belong to the same paragraph of running text",
            false="The two lines belong to different paragraphs or different pieces of content",
        ),
    )

Pass 2 — classify (the whole specification)

"These three dicts, plus the step question's true/false criteria inside classify_questions below, are the entire specification of the classifier. There is no other logic. To adapt the pipeline to your own documents, edit these descriptions."

TYPE_CRITERIA = {
    "heading": "A short label or title that names the document or the section that follows it - not a full sentence of content",
    "paragraph": "Running prose: one or more complete sentences of explanatory or narrative text",
    "list_item": "One entry in a list of parallel items - an ingredient, a feature, a task, an attendee; reads as one of several sibling entries",
    "quote": "Words attributed to a person or source - quoted speech, a citation, an excerpt someone else wrote",
    "code": "Computer code, a shell command, terminal output, or a config snippet meant to be read verbatim",
    "callout": "A warning, tip, or important note that interrupts the flow to flag something the reader must not miss",
}
HLEVEL_CRITERIA = {
    "title": "The title of the whole document",
    "section": "A major section heading within the document",
    "subsection": "A minor heading nested under a section",
}
CALLOUT_CRITERIA = {
    "note": "Neutral extra information the reader should be aware of",
    "tip": "A helpful suggestion or shortcut that makes things easier",
    "warning": "A caution about something that can go wrong or cause harm",
}

HEADING_MAX_CHARS = 90  # longer blocks can't render as headings, so don't ask


def classify_questions(texts: list[str]) -> dict:
    questions = {}
    for i, text in enumerate(texts):
        bid = block_id(i)
        questions[f"type_{bid}"] = Choice(
            instructions=f"What kind of content is block {bid}?", criteria=TYPE_CRITERIA
        )
        if len(text) <= HEADING_MAX_CHARS:
            questions[f"hlevel_{bid}"] = Choice(
                instructions=f"As a heading, what level would block {bid} occupy in this document's structure?",
                criteria=HLEVEL_CRITERIA,
            )
        questions[f"step_{bid}"] = Noul(
            instructions=f"Is block {bid} an instruction in a sequence where the order of the items matters?",
            criteria=NoulCriteria(
                true="It is one step of a procedure - the items around it must happen in order",
                false="Order is irrelevant - it is a loose collection, or not a list item at all",
            ),
        )
        questions[f"callout_{bid}"] = Choice(
            instructions=f"What kind of aside is block {bid}?", criteria=CALLOUT_CRITERIA
        )
    return questions

Why companion questions are asked speculatively: "The types are not known yet, and waiting for them would mean a third round trip, so the companion questions are asked up front in the same request. Most of these answers are never read: the step probability of a paragraph means nothing and is simply ignored. An extra question adds little, since the state is most of the tokens and is sent once either way, while an extra round trip adds a full request of latency." See Speculative fan-out.

Combining logic in code

Setup and pass 1:

import os, re, urllib.request
from pathlib import Path
from time import perf_counter
from typesafe_sdk import Choice, Noul, NoulCriteria, TypeSafeClient

TYPESAFE_MODEL = "jev-1.12"
PRICE = (0.042, 0.00)  # $ per 1M tokens (input, output); TypeSafe jev-1.12 as of 2026-09
client = TypeSafeClient(api_key=os.environ["TYPESAFE_API_KEY"], timeout=120.0)


def stitch() -> dict:
    questions = {line_id(i): join_question(i) for i in range(1, len(LINES)) if not LINES[i]["gap"]}
    started = perf_counter()
    response = client.system_one(
        state=tag(LINES, "L"), questions=questions, model=TYPESAFE_MODEL
    )
    return {
        "joins": [
            response.answers[line_id(i)].noul if line_id(i) in response.answers else 0.0
            for i in range(len(LINES))
        ],
        "seconds": round(perf_counter() - started, 2),
        "usage": [response.usage.input_tokens, response.usage.output_tokens],
    }

The merge, with the punctuation-dependent cutoff:

JOIN_AFTER_DANGLING, JOIN_AFTER_TERMINAL = 0.2, 0.5


def ends_terminal(text: str) -> bool:
    return re.search(r'[.!?:;…]["\')\]]*$', text) is not None


def merge(joins: list[float]) -> list[dict]:
    blocks = []
    for i, line in enumerate(LINES):
        bar = (JOIN_AFTER_TERMINAL if i and ends_terminal(LINES[i - 1]["text"])
               else JOIN_AFTER_DANGLING)
        if blocks and not line["gap"] and joins[i] >= bar:
            blocks[-1]["text"] += " " + line["text"]
            blocks[-1]["lines"].append(i)
        else:
            blocks.append({"text": line["text"], "lines": [i], "gap": line["gap"]})
    return blocks

Pass 2 and rendering:

def classify(texts: list[str], gaps: list[bool]) -> dict:
    tagged = tag([{"text": t, "gap": g} for t, g in zip(texts, gaps)], "B")
    questions = classify_questions(texts)
    response = client.system_one(state=tagged, questions=questions, model=TYPESAFE_MODEL)
    judgments = []
    for i in range(len(texts)):
        bid = block_id(i)
        type_answer = response.answers[f"type_{bid}"]
        hlevel = response.answers.get(f"hlevel_{bid}")
        judgments.append({
            "type": type_answer.choice,
            "confidence": type_answer.confidence,
            "probabilities": type_answer.probabilities,
            "hlevel": hlevel.choice if hlevel else "section",
            "step": response.answers[f"step_{bid}"].noul,
            "callout": response.answers[f"callout_{bid}"].choice,
        })
    return {"judgments": judgments, "n_questions": len(questions),
            "usage": [response.usage.input_tokens, response.usage.output_tokens]}


STEP_THRESHOLD = 0.5
HEADING_MARK = {"title": "#", "section": "##", "subsection": "###"}
CALLOUT_MARK = {"note": "NOTE", "tip": "TIP", "warning": "WARNING"}


def to_markdown(blocks: list[dict]) -> str:
    groups = []
    for b in blocks:
        if b["type"] in ("list_item", "code") and groups and groups[-1][0] == b["type"]:
            groups[-1][1].append(b)
        else:
            groups.append((b["type"], [b]))
    parts = []
    for kind, items in groups:
        if kind == "list_item":
            ordered = sum(b["step"] for b in items) / len(items) >= STEP_THRESHOLD
            parts.append("\n".join(
                f"{n + 1}. {b['text']}" if ordered else f"- {b['text']}"
                for n, b in enumerate(items)
            ))
        elif kind == "code":
            parts.append("```\n" + "\n".join(b["text"] for b in items) + "\n```")
        elif kind == "heading":
            parts.append(f"{HEADING_MARK[items[0]['hlevel']]} {items[0]['text']}")
        elif kind == "quote":
            parts.append(f"> {items[0]['text']}")
        elif kind == "callout":
            parts.append(f"> [!{CALLOUT_MARK[items[0]['callout']]}]\n> {items[0]['text']}")
        else:
            parts.append(items[0]["text"])
    return "\n\n".join(parts) + "\n"

Ordered-vs-bulleted is a group decision: the mean of the items' step probabilities against STEP_THRESHOLD = 0.5. "That threshold is a group-level decision no single question asked directly."

Results the cookbook reports

Model jev-1.12. Pass 1: 16 pair questions, one request, 0.32s → 28 lines -> 17 blocks (11 line breaks healed). Pass 2: 62 questions about 17 blocks, one request, 0.51s.

Per-block classification (abridged from the cookbook's table):

block type conf companion used
B000 heading 0.99 level=title
B002 heading 0.75 level=section
B004 code 1.00 –
B006 paragraph 0.43 –
B007–B009 (team names) list_item 0.99 / 1.00 / 0.99 step=0.15 / 0.16 / 0.12
B011–B013 (to-dos) list_item 0.98 / 0.99 / 0.92 step=0.86 / 0.87 / 0.90
B014 callout 0.65 kind=warning
B015 quote 0.99 –

The step probabilities do the work: the three to-dos near 0.9 render as a numbered list, the three team names near 0.1 as bullets, and the unmarked warning became > [!WARNING]. "Every word above is from the input. The pipeline only chose boundaries, types, and markup."

Where the join thresholds come from

Per-line join probabilities landed "in two separate bands: line breaks that split a sentence score 0.39 and up, breaks the author meant score close to zero." But the cutoff depends on the previous line's punctuation, "a fact code can read directly": true continuations scored as low as 0.39 (make the switch for real.), so a flat 0.5 would break paragraphs; meanwhile L015| The platform team follows a colon and scores 0.22, which would clear a flat 0.2 and swallow the list into its introducing sentence. "No single threshold works for both cases; once code checks the punctuation first, the two bands separate."

Why "mid-sentence" and not "same paragraph"

Same document, same request shape, only the wording changed:

               mid-sentence  same paragraph
L015  The platform team                        0.22         0.77
L016  The web client team                      0.11         0.81
L017  Whoever still owns the release tooli     0.12         0.78
L020  Delete the old build cache directory     0.08         0.88
L021  Run the doctor script and fix anythi     0.05         0.91

blocks after merge: 17 (mid-sentence) vs 12 (same paragraph)

The lesson, quoted: "When a judgment call feeds a threshold, the question should name the narrowest fact that decides it. Here the wording is the difference between 17 blocks and 12." See Writing instructions and criteria that Jev reads correctly.

The lowest-confidence block

"The cutover touches three teams, so check whether you are on this list before you plan anything for Monday:"
confidence 0.43: paragraph 0.53, list_item 0.24, callout 0.19

The cookbook's suggested UI treatment: "underline for review any block whose type confidence (the probability behind the winning choice) is under 0.55."

Cost and latency

pass 1  16 questions  0.32s
pass 2  62 questions  0.51s
total   10,211 tokens  0.8s  $0.0003

Source inconsistency: the code output prints $0.0003, while the prose twice states "$0.0015" for the same run. $0.0003 is what PRICE = (0.042, 0.00) and 10,211 input tokens actually produce, so the prose figure appears to be stale (inferred).

Adapting it to a new domain

Gotchas

Related

Sources