Why bolt-on AI doesn't solve hallucinations

In the last twenty-four months, essentially every established legal technology platform has launched an AI feature. The practice management vendors have added AI assistants. The research databases have added AI research products. Document management systems have added AI summarization. The pitch is universally the same: the AI is integrated into your existing workflow, so you get the benefits of generative AI without the risks.

The pitch is wrong. The architecture underneath these bolted-on AI features is the same architecture that produces the Mata v. Avianca failure mode. The vendor brand on the wrapper does not change what the model is doing underneath. And the absence of structural validation, confidence scoring, and tamper-evident audit logs in the legacy platforms' underlying data models means that the AI feature is, in defensibility terms, no safer than the consumer AI tools that have produced the sanctions cases.

This article walks through what bolt-on AI actually is, why it fails to solve the hallucination problem, and what the alternative architecture looks like.

What "bolt-on AI" means

Bolt-on AI is the addition of generative AI capabilities to an existing software product through a layer that sits on top of the existing data model. The product was built for a different purpose — practice management, legal research, document storage. The AI feature is added later, typically as a chat interface or assistant pane, and is connected to the existing data through whatever APIs the original product exposes.

The structural characteristics of bolt-on AI follow from this history.

The underlying model is third-party. With rare exceptions, the legacy platforms do not train their own large language models. They call out to a third-party foundation model provider via API. The "AI" is generic AI, repackaged inside the vendor's interface.

The data model wasn't designed for AI access. The case management database was designed to track matters. The research database was designed to surface cases. Neither was designed to feed a generative model in a structured way that would allow output validation. When the AI feature retrieves information from the underlying database, it does so through generic APIs that return text — text that the model then incorporates without further structural awareness.

Validation, where it exists, runs after generation. A vendor that wants to claim its AI feature "validates citations" typically does so by post-processing the model's output to check whether cited cases exist in some database. This is meaningfully different from architecturally requiring that the model only produce citations that have already been verified. The post-hoc check can fail. The model can produce a citation; the check can be skipped or unreliable; the citation reaches the user.

The audit log is whatever the underlying product had before. Legacy practice management platforms keep activity logs. Legacy research platforms keep search history. Neither was designed to produce a tamper-evident, cryptographically signed record of an AI session. When the vendor adds an AI feature, they typically log the interaction in the same way they've always logged activity — as a database row that can be queried but isn't, in any rigorous sense, evidentiary.

Why each of those choices matters

The structural decisions that define bolt-on AI map directly to the four failures that produce sanctions cases. We covered these in detail in the DDEAS prevention article, but they're worth restating here in the bolt-on context.

Failure 1: No structural source validation. If the model generates first and validates second (or doesn't validate at all), the hallucinated citation can reach the lawyer. This is the Mata v. Avianca failure exactly. Whether the model is wrapped in a vendor's interface or running directly on the vendor's website doesn't change the underlying behavior. The model has the capacity to invent. The vendor's interface either prevents that invention from reaching the user or it doesn't. Bolt-on architectures generally don't.

Failure 2: No confidence scoring tied to the output. A consumer AI tool produces output with no confidence signal. A bolt-on AI feature, in nearly every implementation we have analyzed, also produces output with no confidence signal — or with a coarse "trust score" that isn't tied to the specific authorities or facts in the output. The lawyer reading the output has no way to distinguish high-certainty paragraphs from low-certainty paragraphs. The verification burden falls on the lawyer's intuition, which is the very thing the AI was supposed to relieve.

Failure 3: No cryptographic audit trail. The vendor's log can show that the lawyer asked a question and received a response. It typically cannot show, in evidentiary form, what authorities the model considered, what was retrieved from the database, what was filtered out, and what made it into the final output. The supervising partner has to trust that the log is complete. The court, in a sanctions challenge, has to trust the same. Without cryptographic signing and chaining, the log is at best a record and at worst contestable testimony.

Failure 4: No tamper-evident chain. Even when a vendor's log is complete, it is generally editable. Database rows can be modified. Logs can be reconstructed. The mathematical property that secures evidence — that altering a historical record breaks a verifiable hash chain — is absent in essentially every legacy platform we have analyzed, including the ones with newly added AI features.

This is not a vendor problem. It is an architecture problem. The legacy platforms cannot retrofit cryptographic audit chains without rebuilding their data models. The bolt-on AI feature inherits the architecture it sits on top of.

The marketing layer vs. the architecture layer

When you read the marketing materials for bolted-on AI features, the language is generally careful. The features are described as "AI-assisted research" or "AI-enhanced drafting" or "AI-powered insights." The vendor's representations stop short of claims that the AI is verified, defensible, or sanctions-proof. This is appropriate, because the underlying architecture doesn't support those claims.

What the marketing does claim, often, is integration. The AI is "integrated with your existing workflow." It "works inside the platform you already use." It "leverages your matter data." These statements are true. They are also irrelevant to the defensibility question. Integration is a usability feature. It is not an ethics feature.

The legal media coverage of these products has, in our reading, been generally too credulous about the difference. The headline coverage of the major legacy platforms' AI products has framed them as solving the hallucination problem because they are "built on" or "trained on" a particular vendor's authority database. The phrase "built on" is doing the work in those framings. "Built on" doesn't mean "structurally validated against." A model that has been trained on a large corpus of cases is not the same thing as a model whose outputs are checked against an authoritative database before reaching the user.

What the AI actually generates

To understand why this matters concretely, consider what happens when a lawyer asks a bolted-on AI feature to draft a section of a motion.

The lawyer types a request. The interface sends the request to the underlying foundation model — typically with some additional context drawn from the vendor's database. The model generates a response. The response contains paragraphs of legal argument and citations to authority. The interface displays the response to the lawyer.

Somewhere in that pipeline, the citations may or may not have been checked against the vendor's case database. If they were checked, the check happened in one of two ways: (1) the vendor's database was provided to the model as context, and the model was instructed to cite from it, or (2) the model's output was scanned after generation to verify citations exist.

Neither is structural validation in the sense that prevents Mata v. Avianca. In approach (1), the model is still capable of inventing — it can be instructed to use the provided context but it is not technically constrained to do so. Foundation models follow instructions probabilistically, not absolutely. In approach (2), the post-hoc check can succeed or fail; if it fails, the hallucinated citation reaches the user. In either case, the structural posture is different from what Mata requires.

A defensibly architected AI feature would do something different. It would generate output through a pipeline where every citation, before it can be incorporated into the response, has been independently verified against authoritative sources. Citations that do not pass verification are not in the model's output stream — not because the model was instructed to avoid them, but because the architecture does not allow them to flow into the response. The model, in this architecture, is not the source of authoritative claims. It is the writer of language around claims that have been independently sourced.

This is the architectural distinction. It is the distinction between a model that may produce verified citations and a system that can only produce verified citations.

Why the legacy platforms can't easily fix this

If the fix is architectural, the obvious question is: why don't the legacy platforms just retrofit the architecture?

The answer is that retrofitting an architecture is harder than designing it from scratch.

Consider what would be required for a major legal research platform to genuinely make its AI feature defensible. The platform's existing case database was designed to be queried by humans through a search interface. To support structural AI validation, the database would need to be reorganized to expose authority in machine-verifiable form, tied to each citation by stable identifiers, with current-good-law overlays that can be programmatically queried. The AI feature's request pipeline would need to be rebuilt so that every output passes through a validation gate before display. The audit log infrastructure would need to be replaced with cryptographically signed and chained storage. The confidence scoring infrastructure would need to be designed and tied to the new validation layer.

This is not a feature addition. This is a re-architecture. For a platform with millions of paying customers depending on the existing system to keep working, re-architecting is genuinely difficult. It is easier — and from a product roadmap standpoint, more rational in the short term — to add the AI feature on top of what exists.

The result is a market in which the platforms with the largest customer bases are the ones least structurally positioned to deliver defensibly architected AI, and the new entrants without legacy customers are the ones positioned to do so.

What a defensible architecture looks like

Defensible AI architecture, for legal practice specifically, has four structural requirements:

One — validation as a gate, not a check. Citations and authority assertions must pass through validation against current authoritative sources before they appear in the model's output. The validation is not an after-the-fact verification; it is a constraint on what the model is permitted to produce.

Two — confidence scoring tied to the substance. Every output should carry a confidence signal that reflects the strength of the underlying authority and the consistency of supporting sources. Generic "trust scores" that apply to the whole output are not sufficient; the scoring needs to be paragraph-level or claim-level.

Three — cryptographic, chained audit log. Every action the AI takes — every query, every retrieval, every output — needs to be logged in append-only form, signed, and chained so that the log itself is tamper-evident and authenticatable.

Four — unified data model. The data the AI operates on (matter information, documents, authority, prior work product) must be part of a single coherent data model rather than retrieved through ad-hoc integrations across multiple legacy databases.

A platform that delivers all four is structurally different from a platform that delivers two or three of them. The legacy platforms with bolt-on AI features generally deliver zero or one. The structural gap is large.

What Discover Docket is doing differently

DDEAS — the Discover Docket Ethical and Accountability Standards framework — implements all four requirements as architectural features rather than feature additions.

Validation is a gate. JILL cannot return a citation that has not been validated against current authoritative sources for the jurisdiction your matter sits in. The validation runs on every output, every time. There is no configuration option to disable it.

Confidence scoring is paragraph-level. Every section of a JILL-drafted document carries a confidence score. The scoring is produced by independent models that evaluate output against source material, consistency of supporting authority, and freshness of underlying data.

The audit log is cryptographically signed and chained. Every JILL action — query, retrieval, output, score — is recorded in append-only form, signed with a key tied to your firm, and chained into a tamper-evident sequence. The same architecture that secures financial transaction records and digital evidence chains of custody secures DDEAS audit logs.

The data model is unified. JILL operates on a single Discover Docket data structure that includes matter information, documents, authority, deadlines, and prior work product. There is no integration layer reconciling differing models. The lawyer sees one coherent representation because there is only one.

The reason this works is that we built it this way from the start. We did not add AI to an existing platform. We built the platform around the architecture AI needs to be defensible.

If a vendor tells you their AI is built on top of their existing platform, the right question to ask is: was the platform designed for what the AI now needs to do? In essentially every case, the honest answer is no. The platform was designed for what the platform did before. The AI was added because the market demanded it. And the lawyer using that AI is in the Mata position, with the brand of the platform underneath, but the architecture is the architecture that produced Mata.

Read the DDEAS framework →

See the platform we built around it →