The Trust-Laundering Machine: When Institutions Sign the Slop
October 5, 2026 by Asif Waliuddin
A May 2026 audit of the scientific record put the conservative 2025 count of hallucinated citations at 146,932, each one sitting in the literature as if it pointed somewhere real.
The dangerous moment is not when AI invents a claim. It is when an institution gives that claim authority.
The signature was real.
The authority underneath it was not.
In June 2026, the Georgia Court of Appeals reviewed a trial-court order in a dispute involving Henry County Schools.
The order had been drafted by an attorney and signed by the court.
It looked like a judicial decision because, procedurally, it was one.
There was only one problem.
The appellate court found that the order misquoted cases, ignored binding precedent, and relied on at least one hallucinated case.
The court called counsel's submission troubling.
It also called the trial court's failure to catch the errors regrettable.
Then it reversed in part, vacated in part, and remanded. (Henry County Schools v. Grant, Ga. Ct. App., June 10, 2026)
The important part of that story is not whether a language model produced the fabricated authority. The Georgia court made no finding that a model produced it, though it cited Slay v. Ross on hallucinated cases and reminded counsel, and lower courts, of their professional obligations in the use of artificial intelligence.
The important part is what happened to the claim after it entered the institution.
A lawyer drafted it.
A court signed it.
The institutional signature changed its status.
What began as unsupported material crossed a boundary and returned to the world carrying judicial authority.
That is the failure mode Episode 4 of The Honor System names trust laundering:
The conversion of an unverified claim into accepted authority through institutional promotion.
The error did not become true.
It became official.
The failure is not generation. It is promotion.
The first three episodes in this series followed synthetic work through an organization.
Episode 1 asked what happens when machine-speed generation outruns human digestion.
Episode 2 named the resulting review debt.
Episode 3 followed provisional information into persistent memory and showed how retrieval can give yesterday's claim authority over today. (The Memory Trust Trap: When Recall Becomes Authority)
Episode 4 crosses the next boundary.
The artifact leaves the realm of provisional work.
Someone signs it.
Files it.
Approves it.
Publishes it.
Indexes it.
Ships it.
The organization has now done more than remember the claim.
It has promoted it.
That moment deserves its own architecture because promotion changes the economics of verification.
Before promotion, the claim is one artifact among many.
After promotion, downstream people and systems are rationally more likely to trust it.
The court signed it.
The partner approved it.
The regulator published it.
The executive accepted it.
The knowledge platform marked it validated.
The model retrieved it from an approved corpus.
Every one of those signals reduces the probability that the next consumer will repeat the original verification.
That is how authority compounds faster than evidence.
The promotion boundary
I call the transition the promotion boundary.
A promotion boundary is the point where provisional content gains permission to exercise institutional authority.
Every organization has them, even when nobody has named them.
A pull request gets merged.
A draft becomes policy.
A recommendation becomes a decision.
A research memo becomes an executive briefing.
A provisional record enters the approved knowledge base.
A generated answer becomes customer-facing guidance.
A legal draft becomes a filed brief.
A proposed order becomes a court order.
Before the boundary, skepticism is expected.
After the boundary, trust is inherited.
That inheritance is useful. Civilization would be impossible if every person had to independently re-verify every fact in every institutional record before using it.
But AI changes the volume and speed at which provisional claims can reach those boundaries.
That creates a new design requirement:
The right to cross a promotion boundary should depend on the evidence attached to the artifact, not merely on the workflow state attached to it.
Otherwise "approved" becomes an aesthetic property.
The signature fallacy
The Georgia case exposes a second mechanism:
the signature fallacy.
The signature fallacy is the assumption that an approval marker proves the underlying evidence was checked.
It does not.
A signature proves that someone signed.
An approval proves that someone approved.
A green status proves that a workflow recorded green.
None of those events automatically prove that the signer inspected the evidence beneath every load-bearing claim.
This sounds obvious until you look at how institutions actually operate.
We routinely use status as a proxy for verification because, historically, status was expensive to earn.
A court order required legal work.
A published report went through editors.
An approved architecture decision passed expert review.
A signed executive memo survived layers of scrutiny.
Those expectations became embedded in the meaning of the artifact.
AI does not eliminate those processes.
It changes their throughput.
When production accelerates faster than verification, the old status symbols remain while the evidence density underneath them can thin out.
The institution still looks like an institution.
That is why trust laundering is hard to see.
The Ninth Circuit drew the boundary explicitly
On June 3, 2026, the U.S. Court of Appeals for the Ninth Circuit issued a sanctions order involving briefs containing nonexistent cases, misattributed quotations, and "gross misrepresentations of real cases."
The court made an unusually useful distinction.
It said the violation did not occur merely because generative AI may have been used during research or drafting.
The responsibility boundary was signing and filing.
The court explained that a lawyer's signature on a filing is an attestation that the signer reviewed it and is responsible for the accuracy of its contents.
Two attorneys were sanctioned, but only one of them, Sethi, signed the briefs. Each was sanctioned $2,500 and suspended from practice before the Ninth Circuit for six months, and the court said the suspension was owed to their repeated failure of candor about where the errors came from.
For two years, the two attorneys and every attorney at their firm must include in all future filings a statement, made under penalty of perjury, addressing whether generative AI was used, naming the tool, and certifying that the signing attorney personally reviewed the filing and that every citation and quotation refers to existing authority.
Sethi filed a petition for certiorari with the U.S. Supreme Court on September 1, 2026. As of October 4, that petition remains pending. (Supreme Court docket No. 26-296)
The legal specifics matter less here than the architecture.
The Ninth Circuit placed accountability at the promotion boundary.
Research can be provisional.
Drafting can be assisted.
But the signature changes the claim's institutional status.
The institution owns what it promotes.
Verification belongs to the final artifact
Connecticut supplied an even cleaner lesson in an order dated July 31, 2026.
An attorney researched his cases through LexisNexis and verified citations in his drafts.
Then he used ChatGPT to improve the organization and writing.
The AI editing pass added or altered citations.
Neither the attorney nor the firm partners who reviewed the final documents re-verified those citations before filing them.
Approximately seven erroneous and unverified citations reached the Connecticut Supreme Court. (Connecticut Supreme Court order, July 31, 2026)
That sequence destroys a comforting assumption:
"We verified the document."
Which document?
The draft before the AI edit?
The version after the edit?
The copy that was actually signed?
Evidence verification is not an abstract property of an idea.
It belongs to a specific artifact version.
If the artifact changes after verification, the verification receipt can expire.
This is why the right unit of trust is not the document name.
It is the claim, evidence, and exact promoted version.
A source check performed three revisions ago cannot certify words that did not exist yet.
The problem is no longer anecdotal
As of July 23, while Episode 4 was in production, Damien Charlotin's AI Hallucination Cases Database listed 1,796 judicial and tribunal decisions.
By September 1 it had passed 2,000.
The database was updated October 4, 2026, and now lists 2,145 decisions, 1,473 of them from U.S. courts. (AI Hallucination Cases Database)
Those numbers need to be handled carefully.
This is not an incidence rate for legal AI use.
It does not tell us what percentage of AI-assisted filings contain fabricated authorities.
It is a curated corpus of decisions in which courts or tribunals found, or in some cases clearly implied, reliance on hallucinated material. It also includes some decisions where AI use was alleged but not confirmed.
The denominator is unknown.
But the direction is no longer a novelty story.
The same failure keeps reaching the institutional boundary.
And courts are increasingly responding not to AI use itself, but to unverified promotion.
Science has a parallel publication problem
Law gives us unusually clean evidence because fabricated authorities are externally checkable.
Scientific publishing exposes the same structural question at a different institutional boundary.
A May 2026 preprint audited 111 million references across roughly 2.5 million papers from arXiv, bioRxiv, SSRN, and PubMed Central.
The authors reported a sharp rise in nonexistent references after widespread LLM adoption, with what they call a conservative estimate of 146,932 hallucinated citations in 2025 alone. (arXiv 2605.07723)
That number should not be treated as settled fact.
The paper is a preprint. The authors label these citations hallucinated, but as I read it, the difficult methodological issue is attribution: a nonexistent citation can result from an LLM hallucination, but it can also result from ordinary citation mistakes, indexing gaps, title variation, or database problems.
The study is still useful because of the institutional mechanism it tests.
A reference can survive authoring.
Survive manuscript preparation.
Survive automated checks.
Survive editorial handling.
And enter the scientific record.
The authors themselves write that preprint moderation and journal publication processes "capture only a fraction of these errors."
Once published, future researchers, or future AI systems, may encounter that reference as part of an authoritative literature graph.
Again, the important transition is not the original error.
It is institutional promotion.
Authority compresses verification
This is the part leaders should care about most.
Institutions are useful because they compress verification.
A reader does not reproduce a clinical trial before reading the paper.
A developer does not reconstruct every architectural decision before using the runbook.
An executive does not inspect every source behind every number in a board deck.
A judge does not independently relitigate every precedent cited in every filing.
Instead, we rely on layered institutional trust.
Authors check.
Reviewers review.
Editors edit.
Approvers approve.
Signers sign.
That division of labor is not a weakness.
It is how complex organizations function.
But it has one hidden dependency:
Each layer assumes the prior layer performed the verification its role implies.
AI breaks that assumption when it increases artifact production without increasing evidence-bearing review.
The downstream consumer sees the old authority signal.
The upstream verification may no longer exist at the same density.
That is trust laundering.
The cure is not another disclaimer
A line that says "AI may have been used" does not solve this problem.
Neither does a blanket instruction to "verify AI output."
Both can be useful.
Neither establishes that verification occurred for the claim now being promoted.
The control has to travel with the evidence.
Episode 4 introduces three operating requirements.
1. Claim-level evidence receipts
A document-level source list is not enough for consequential claims.
Each load-bearing claim should be able to answer:
- What exactly is being asserted?
- What evidence supports it?
- What version of the evidence was checked?
- Who or what performed the verification?
- When did the verification happen?
- What was the result?
- What limitations remain?
This is not about attaching ten thousand receipts to casual prose.
It is about making evidence granular enough that a high-impact claim can survive downstream reuse without losing its proof.
A citation alone is not an evidence receipt.
A citation proves that a source was named.
It does not prove that the source exists, is current, or actually supports the claim being made.
2. Sign the proof set, not merely the prose
An approver should not only approve the words.
The approver should approve the evidence state associated with the consequential claims inside those words.
That changes the meaning of approval.
Instead of:
"I read this document."
the promotion event becomes closer to:
"I am approving this version of these claims with this evidence set at this time."
That is a much stronger object.
It is also more automatable.
Machines can check whether required receipts exist.
They can verify hashes and versions.
They can detect missing evidence.
They can flag when a cited source changed.
They can require independent review for certain claim classes.
The human does not need to manually inspect every low-risk sentence.
The system needs to know which assertions are load-bearing and whether they have earned promotion.
3. Retractions must propagate
Institutional trust creates a downstream problem when something is later found to be wrong.
A correction at the source is not enough.
If the original claim was copied into a knowledge base, cited in a decision, embedded into a retrieval store, summarized into agent memory, or used to generate another artifact, then the correction has dependents.
A governed system should be able to ask:
Where did this claim travel after we promoted it?
Then a retraction can propagate.
The promoted claim can be revoked.
Dependent artifacts can be flagged.
Agents can stop retrieving it as current authority.
Humans who relied on it can be notified when appropriate.
Without that mechanism, a correction can coexist indefinitely with the institutional descendants of the error.
The source gets fixed.
The organization stays wrong.
I hit the same mechanism internally
Inside NextGen AI, I encountered a smaller version of this problem.
A true approval claim had a source attached to it.
At first glance, that looked like evidence binding.
But the linked record was the author's correction, not the independent reviewer's actual verdict.
The claim happened to be true.
The receipt was wrong.
That distinction matters because truth and proof are separate properties.
A system cannot claim evidence governance merely because its conclusions are usually correct.
It has to preserve the evidence that justifies those conclusions.
I fixed the missing receipt.
But the lesson is more important than the patch:
A source reference is not enough. The evidence has to entail the claim.
That is why my current posture remains practice, not product.
Evidence binding is partial and workflow-specific.
Auditability is stronger on selected governed paths.
I do not claim universal tamper-evident provenance.
The Honor System applies to me too.
Published is not proven
By the time an unsupported claim reaches a court order, published report, policy document, approved architecture record, or validated knowledge store, the difficult part has already happened.
The claim borrowed the institution's reputation.
That is why downstream consumers stop asking whether the original evidence was checked.
The artifact carries the answer implicitly:
Of course it was checked. Look who signed it.
That assumption was always imperfect.
At machine-speed production volume, it becomes dangerous.
The enterprise response cannot be permanent skepticism toward every institutional artifact.
That would destroy the value of institutions.
The answer is to rebuild the compression mechanism so authority carries evidence with it.
Promotion should mean something observable.
Approval should have a receipt.
A signature should identify the proof set it attests to.
And a retraction should be able to find its descendants.
Then institutional trust can scale without becoming an honor system.
Because the signature can be real.
The document can be real.
The institution can be real.
And the authority underneath all three can still be synthetic.
Published is not proven.
Never let synthetic work become organizational memory without evidence.
This article accompanies Episode 4 of AI Unveiled (The Honor System), "The Trust-Laundering Machine: When Institutions Sign the Slop," published August 6, 2026. It follows an unsupported claim across the institutional promotion boundary and introduces claim-level evidence receipts, proof-set approval, and propagating correction. Listen to Episode 4 on Apple Podcasts.
Episode 3, The Memory Trust Trap: When Recall Becomes Authority, asked what happens when memory returns with authority. Episode 4 asks what happens when an institution signs that authority and sends it downstream.
Once an organization understands that signatures and approvals are not enough, the next question becomes unavoidable: what would a control look like if it could actually stop unverified work from crossing the boundary? Continue with Episode 5, Policy Is Not a Control: What Europe Just Taught America About AI Governance.