Decisions, not text: Jev in a data pipeline

Most of the work in a data pipeline is not writing. It is deciding. Are these two records the same company? Is this sentence a claim or a quote? Does this statement contradict that filing, or describe a change? Which of five categories does this document belong to? These are classification decisions, made millions of times, and most of them are still sent to models designed to generate text.

Jev, from TypeSafe AI, is built for the other case. It does not write. It answers typed questions with probabilities. This note looks at how such a model works, where it fits in a pipeline that resolves entities and compares records, and how to use it without trusting it blindly.

Generation versus decision

Structured outputs have made generative models reliable at returning a valid label: constrained decoding guarantees that the answer matches a schema. What they return, though, is a single answer. Its uncertainty, when available at all, comes from token probabilities that were never trained to mean anything about the decision.

A decision model makes the uncertainty the product. The application declares the question and its possible answers; the model returns a probability for every option, by design and in every response. TypeSafe describes Jev as a “System One” model, a reference to fast, intuitive judgement, and says it is trained with a method it calls Reinforcement Learning for Calibrated Decisions. The other difference is economic: no tokens are generated, so cost and latency are set almost entirely by the length of the input.

The interface

A request contains two things: a state, the text to judge, and a set of named questions. Each question has a type and instructions.

• noul: a yes-or-no judgement, answered with a single probability between 0 and 1.
• choice: one label among a declared set, answered with the chosen label, a probability for each option and a confidence value.
• score: a position on a declared ordinal scale, answered with a numeric score, a probability for each level and a confidence value.

Several questions can be asked about the same state in one call, and they are evaluated together. Input is text only. A choice question is limited to 255 options. Billing is on input tokens; output is free.

An illustrative request for entity resolution, with fictional records:

{
  "model": "jev-1.13.0",
  "state": {
    "record_a": "Northvale Holdings Ltd, director: J. A. Moreau, registered 2014, 12 Harbour Street",
    "record_b": "Northvale Holding, administrateur : Jean-Alain Moreau, immatriculée en 2014"
  },
  "questions": {
    "same_entity": {
      "type": "noul",
      "instructions": "Do both records describe the same legal entity?"
    },
    "relation": {
      "type": "choice",
      "instructions": "What is the relationship between the two records?",
      "criteria": {
        "same": "The same legal entity.",
        "parent_subsidiary": "One controls the other.",
        "related": "Distinct entities sharing people or addresses.",
        "unrelated": "No evident relationship.",
        "insufficient": "Not enough information to decide."
      }
    },
    "name_similarity": {
      "type": "score",
      "instructions": "How closely do the two names match, ignoring legal suffixes and language?",
      "criteria": ["Different", "Partly similar", "Equivalent"]
    }
  }
}

And the shape of the response, with illustrative values:

{
  "model": "jev-1.13.0",
  "answers": {
    "same_entity": { "type": "noul", "noul": 0.91 },
    "relation": {
      "type": "choice",
      "choice": "same",
      "probabilities": { "same": 0.88, "parent_subsidiary": 0.04, "related": 0.05, "unrelated": 0.01, "insufficient": 0.02 },
      "confidence": 0.84
    },
    "name_similarity": {
      "type": "score",
      "score": 1.9,
      "probabilities": { "0": 0.01, "1": 0.08, "2": 0.91 },
      "confidence": 0.89
    }
  }
}

Two fields deserve care. The published documentation does not specify how the confidence value is computed, nor how the fractional score is derived from the level probabilities. Both are best treated as additional signals, to be calibrated against labelled data like the probabilities themselves, rather than read at face value.

Two design choices in the request matter as much as the model. An explicit “insufficient” option gives the model a way to abstain instead of forcing a guess. And asking related questions together (same entity, nature of the relation, name similarity) provides internal consistency checks: a high “same entity” probability alongside a low name-similarity score is itself a signal worth inspecting.

Where it fits

In a pipeline that reads documents, resolves entities and compares records, typed decisions appear at almost every stage.

Record linkage. After blocking has reduced the candidate pairs to those sharing a name, a date, an address or an identifier, each remaining pair needs a verdict. This is the largest volume of decisions in the pipeline, and the one where cost per call matters most.

Claim typing. Each extracted statement must be classified: numerical, positioning, forward-looking or biographical; asserted, hedged, conditional or denied. These are choice questions over fixed vocabularies.

Contradiction triage. Pairs of claims about the same subject and period can be sorted into entailment, contradiction, change over time or unrelated, before any expensive review.

Routing. Deciding which documents need a full extraction pass, which are duplicates, and which should be escalated to a human or to a generative model for a written explanation.

The pattern is a division of labour. Fast typed decisions handle the volume; generative models handle the cases that need language; people handle the cases that need judgement.

The usual alternative for this volume is a small classifier trained in-house, such as a fine-tuned cross-encoder. It is cheap to run and can be calibrated well, but it needs labelled data for every task and retraining whenever the task changes. A decision model changes task by changing the question, which matters in a pipeline where the questions evolve faster than the training sets.

From pairs to entities

Pairwise verdicts are not yet entities. They can be inconsistent: A matches B, B matches C, and A does not match C. If pairs are simply chained together, a few false matches are enough to merge distinct entities into one large, wrong cluster.

A clustering step is therefore required, turning pairwise probabilities into consistent groups. Connected components above a threshold are simple but prone to this chaining effect. Correlation clustering and hierarchical agglomerative clustering, which weigh all the pairwise evidence within a group, are more robust. Pairs whose verdicts conflict with the cluster they would join are good candidates for review.

Thresholds and calibration

Probabilities are only useful if they mean what they say. A model is calibrated if, among all the decisions it makes with probability 0.9, about 90% are correct.

Calibration has to be measured, not assumed, on a labelled sample from the pipeline’s own data: the probability range is split into bins, and the observed accuracy in each bin is compared with the predicted probability. The standard summaries are the reliability diagram, the expected calibration error and the Brier score. Calibration measured on one domain does not transfer automatically to another; registry records, transcripts and press articles should each be checked.

When probabilities are miscalibrated, they can be corrected without touching the model. Temperature scaling fits a single parameter on a labelled sample; isotonic regression learns a monotonic mapping from predicted to observed probability. Both are cheap and should be re-fitted per domain and per model version.

Thresholds then follow from the cost of errors. In entity resolution, a false merge (two different people treated as one) is usually more damaging than a false split, because it contaminates every downstream link. The practical setting is two thresholds, not one: above the upper threshold, accept; below the lower one, reject; in between, route to review. The width of the band is a trade-off between precision and review workload.

The decisions themselves are then evaluated with the metrics of entity resolution: pairwise precision and recall at each threshold, and cluster-level metrics such as B-cubed once pairs have been grouped into entities.

Model versions matter throughout. A threshold or a calibration mapping fitted on one version is not guaranteed to hold on the next. Pinning a specific version, rather than an alias that changes over time, keeps decisions reproducible; moving to a new version means re-running the calibration check first.

Cost at scale

Because pricing is on input tokens only, the cost of a decision is set by the size of the state. With TypeSafe’s published price of $0.042 per million input tokens, a pair described in about 300 tokens costs roughly a thousandth of a cent. Deciding 50 million candidate pairs, that is 15 billion tokens, costs about $630. TypeSafe reports end-to-end latencies of roughly 70 to 500 milliseconds per call.

These figures change what is economically reasonable. It becomes possible to ask more questions per pair, to re-decide past pairs after a model update, and to run redundant checks on the decisions that matter most.

Limits

The model’s guarantees are narrower than they may sound.

• Schema, not truth. The output always matches the declared format; it can still choose the wrong option. “No malformed output” is not “no errors”.
• Text only. Images, scanned pages and audio must be converted upstream.
• Bounded choices. With at most 255 options per choice question, open-ended matching against a large list of candidates has to be restructured, for example into pairwise noul questions after blocking.
• Vendor figures. The speed, cost and accuracy claims come from TypeSafe. There were no independent evaluations at launch, and no weights or detailed paper with which to reproduce them.
• Instructions are part of the model. Wording, option descriptions and the presence of an abstention option change the distributions. They should be versioned and evaluated like code.
• Data location. Every request sends the state to a hosted service, currently operated from the United States. When the state contains personal data covered by the GDPR, the place of processing and the contractual terms with the provider need to be checked.

Summary

A typed decision model treats classification as what it is: a choice among declared options, returned with probabilities. In a pipeline that links records and compares claims, that is most of the work. Used with calibration checks, a clustering step, two thresholds, pinned versions and a human review band, it turns millions of small judgements into decisions that can be measured, audited and repeated.