Jev Integrate

Wiring a System One model into a feature

A System One model answers typed questions in one forward pass. It generates no text. State in, typed answers with calibrated probabilities out. It is an if-statement that can read.

Use it when the decision is narrow, pre-specified, and repeated. Do not use it for anything that needs a written explanation — that is still a job for Claude.

Before anything else: is this actually the right tool

Answer these three. If any is "no", stop and keep the LLM call.

1. Are the possible answers known up front? Choice caps at 255 options.
2. Does the caller need only the label, not the reasoning? If a human reads a justification downstream, you need prose and this is the wrong tool.
3. Is it high volume, or is a person waiting? This is a latency and cost optimisation, not a capability gain. It knows nothing Claude doesn't. On a nightly cron over fifty records it buys you a dependency and nothing else.

Measured on 150 hand-labelled records across three jobs (our Sep 20 2026 run): Jev ties GPT-5.2 at 145/150 and costs 46x less ($0.036 vs $1.64 per 1k records), but end to end it is only 1.7x faster than GPT-4.1-mini — the published 40x-200x is against a 3-329 s multi-step frontier workflow, not one call.

The open reproductions are not drop-in. Same run: Von 1.0.1 (395M) 92/150 (61%), Laya (421M) 62/150 (41%). They collapse onto one class rather than degrading — Von predicted exfiltration 25 times on a 50-command set containing five. A confidence gate does not rescue that: catching Von's errors meant escalating 92% of volume, Laya 100%, against Jev's 8%. Use them only where you have measured them on your own labelled set.

The three question types

"lead_type":  {"type":"choice", "instructions": "...", "criteria": {"opt_a":"desc","opt_b":"desc"}}
"is_urgent":  {"type":"noul",   "instructions": "..."}                       # -> 0.0–1.0
"priority":   {"type":"score",  "instructions": "...", "criteria":["ignore","low","high"]}

Ask every question you need in one call — they all resolve in the same forward pass, so four questions cost roughly what one does.

Response shape (both Jev and Von):

r["answers"]["lead_type"]["choice"]         # the label
r["answers"]["lead_type"]["probabilities"]  # full distribution
r["answers"]["lead_type"]["confidence"]     # use this for gating
r["answers"]["is_urgent"]["noul"]           # 0.0–1.0
r["answers"]["priority"]["score"]           # position on the scale, e.g. 2.41

Workflow

1. Build the labelled set FIRST — 50 records minimum

Non-negotiable, and the single highest-value step. Hand-label real records from the stream you intend to point this at, before writing any criteria. Without it you cannot tell a bad question from a bad model, and the failure is silent — see jev-eval.

2. Write the criteria as if explaining to a new hire

Worst-to-best spread across four wordings of the same questions, 50 records per task (our Sep 20 2026 run):

Same sweep on the command task with the LLMs included: Haiku 4.5 46-48 (4 pts), GPT-4.1-mini 45-50 (10 pts), Jev 44-49 (10 pts), GPT-5-mini 41-49 (16 pts).

Jev is NOT more wording-robust than a small LLM — it swings the same ten points, and Haiku was the steadiest model in the test. Read the FLOOR, not the spread: every hosted model bottoms out at 82-92% and stays shippable, while Von bottoms out at 18% and Laya at 36%. Do NOT read this as "write better criteria and the open model catches up" — an earlier 15-record test concluded exactly that and it was wrong. Richer criteria did not reliably help: on lead triage Von scored 34/50 on the terse wording and 28/50 on the carefully written one. What moves those numbers is sensitivity to surface form, not comprehension, so every future criteria edit is an unannounced regression risk.

Write each option with: what it is, what it is not, and the edge case that tempts a wrong answer. Name the default explicitly when one option should dominate.