The Multi-Model Move You're Already Making
When people say “multi-model architecture,” they usually picture a production pipeline. Extractor, validator, arbiter. Three models in a row, one final answer for the customer. The fancy thing.
Skip the fancy thing for a second.
There is a multi-model practice you are probably already running every day, and it does not have an architecture diagram. It looks like: an operational model — a browsing agent, a scraping agent, something doing real work — does something weird, and you paste its trace into a strong analytical model and ask “what just happened here?”
Why did the browsing agent come back with four irrelevant sites instead of the one I asked for? Why did the scraping agent decide to wander off to a different domain entirely? Why did the operational model just confidently produce output that contradicts the obviously correct answer sitting right there in the source document?
That is a multi-model disagreement workflow. The operational model gave you one story. The analytical model is giving you another. The disagreement between them is the actual diagnostic instrument. Engineers reach for it without naming it.
The instinct behind that move — preserve the disagreement, do not collapse it into a verdict — is what should govern production reconciliation when it eventually lands. Most teams get this backwards. They invest in elaborate production reconciliation logic while leaving the development layer ad hoc, when the development layer is the one where multi-model is already working.
Multi-model practice runs on different rules at different layers of your work, and most discussions of “reconciliation” mix them up.
There is the production layer. Model output is going to a customer or driving a downstream decision. In legal document processing, which is the lens I think in, that means an extracted field a title examiner will rely on, a deed boundary that will land in a closing package. Here, multi-model is about signaling, not voting. Two models disagree on a field, that field deserves human eyes. The system can be smart about how it presents the disagreement, but it does not get to pick a winner. The cost of being wrong is paid by a customer in a regulated workflow, and that cost is structurally higher than the engineering effort to keep a human in the loop.
Then there is the development layer. One model is interrogating another, often across vendor families on purpose. The analytical model, a strong reasoning model, reads what the operational model just did and offers a hypothesis about why. This is multi-model debugging, and it is how a lot of the operational work on a modern AI system actually gets built and tuned. The cost of being wrong is a slightly worse hypothesis or a wasted exploration. Different cost function, different practice.
The mistake people make is treating these two layers like they should run on the same logic. They should not. The production layer is conservative because the customer pays the cost of wrong. The development layer is exploratory because the engineer pays the cost of wrong. What is shared between them is the principle: at both layers, disagreement is the diagnostic instrument, not the bug to silence.
That shared principle is the actual subject of this piece. Everything else is application.
In the production layer, we do not auto-reconcile.
The current customer-review surface is straightforward. Every document the system processes ends up in a structured form interface keyed to its document type. The reviewer edits each field individually, then approves the whole document at the end. It is a real review, not a rubber stamp, but every field is presented with equal visual weight. There is no signal in the UI about which fields the system was uncertain about or which fields two models disagreed on. The presentation is flat.
That flat presentation is what a reconciliation layer changes. Not the humans-approve part. That stays. The direct your attention part.
When we add a reconciliation layer for legal documents, here is the shape it has to take.
It compares structured outputs at the field level. The diff already exists in the pipeline, mechanically. Extracted documents end up as JSON, and disagreement happens field by field, not document by document. Most of the engineering work for “structured disagreement output” is already in place; you do not have to invent it.
It makes the human’s call faster, not in place of their call. Surface both candidate values, the source-text anchors each model latched onto, and what kind of disagreement it looks like: numeric mismatch, name variant, role confusion, semantic disagreement. The layer proposes nothing. It frames. In the existing form-review UX, this means visual differentiation between fields the system is confident about and fields where two models disagreed, with one-click access to both candidates and the source span. The reviewer still touches every field they want to touch. They spend their attention where the system has signaled that attention is warranted.
It does not normalize “trivial” differences upstream of the human. We will come back to that one. It is the load-bearing argument.
And it does not try to vote. Two production extraction models, especially different ones, fail in correlated ways on some fields and uncorrelated ways on others. “Majority wins” rewards the dimensions where they agree for the wrong reasons. “Highest confidence wins” rewards the most overconfident model, which is its own selection bias. Both are reframings of a domain question as an arithmetic one. In legal work, you cannot answer a domain question with arithmetic.
The thread tying these properties together: useful reconciliation preserves the disagreement signal long enough to make a good human decision about it. Bad reconciliation rushes to collapse the signal into a single answer that looks like progress.
The dev-time multi-model practice is where this principle is already paying off, every day. Two patterns are currently in use that exemplify it.
The first is an iteration agent. We have built an internal harness for title work, call it the title agent harness, that sits in the same conceptual category as agentic harnesses like Claude Code: there is a core system prompt that contains the playbook (ordered instructions for what to do, step by step, and what tools are available at each step), and a runtime that executes against it. The playbook is configurable to a customer’s specific process.
The iteration agent runs the harness against reference data multiple times. It makes small adjustments to the playbook each iteration and produces an updated playbook for review. Critically, it is iterating on the instructions, not on customer-facing outcomes. The reference data is the ground truth; the playbook is the lever. It is a closed feedback loop, entirely inside development, where the cost of a wrong tweak is a slightly worse iteration and nothing more.
The second is a flow-review agent. It watches recurring agentic flows. When a run errors out or takes longer than expected, the review agent reads the trace and produces recommendations for things to improve. The recommendations are not applied automatically. A human reads them, judges them, decides what to change.
Both are multi-model architectures, but the multi-model-ness is not between two extraction models inside a single production request. It is between an operational model (the one doing the work in the trace) and an analytical model (the one reading the trace and reasoning about it). The analytical model’s job is to make humans smarter and faster. It is explicitly not in the decision-making seat.
This is the practical answer to “what is multi-model architecture in production today?” It is not extractor-plus-validator-plus-arbiter sitting inside a customer-facing request. It is a running operational layer observed by an analytical layer whose verdict goes to an engineer.
Now, the load-bearing argument. The skeptical reader pushes back: fine, you do not auto-reconcile the high-stakes stuff. But surely you can auto-reconcile the trivial differences. Whitespace. Capitalization. Different date formats that mean the same thing. Leading zeros. Why pull a human into “1,000” versus “1000”?
The legal-document answer:
A stray mark on a scanned page can be the difference between 342 acres and 3.42 acres. There is no LLM that is going to reliably distinguish a deliberate decimal point from a scan artifact every time. The “trivial” decimal-place normalization stops being trivial the instant the underlying mark is ambiguous. And in legal source documents, the underlying mark is ambiguous all the time.
This is the same digit-pattern problem I wrote about in the last piece. 45.1 versus 451, same digits, different meaning. In the single-model framing of that earlier article, it was a confidence problem; one model returning one number could not tell you which way the failure went. In a multi-model framing, the same case becomes a disagreement problem, and the disagreement is the more useful instrument: one model read the dot, one did not, and the system now has something it can do about it. Auto-normalizing the difference erases the signal.
Going upstream does not escape the problem. Even if we worked with a larger title plant that offers an API to structured data, the data in that API was entered by humans in municipalities, and those humans introduce the same class of error and that is not changing anytime soon in municipalities. The “we will just use the canonical source” instinct pushes the problem up one layer; it does not eliminate it.
So the rule is not “we never normalize anything.” The rule is: the criteria for what is safely auto-normalizable in a legal context are themselves so specific that they require the same engineering judgment we already use for the reconciliation question. You do not get to skip the work. You only get to choose which layer of the system pays for it.
That is the real reason production reconciliation in this domain has to keep a human in the seat. Not because reconciliation is technically hard. Because “trivial” is a domain judgment, not an engineering one.
Multi-model is not a production architecture move. It is a discipline that shows up everywhere there are two models in the room — including, most often, between the engineer and the agent that just did something weird. The discipline is the same in both places. Do not optimize the disagreement away.
It is the only diagnostic instrument you have got.