New from the Lab·The Compass — an open moral reasoning standard for AI, tested across frontier modelsExplore →
Production AI Institute · PSF v1.1 open standard
AI Right-To-KnowAI Data Use IndexCheck My AI ToolsPolicy Change WatchAgent ReadinessPublic BenchmarkContactGlobal standard · Worldwide
Incident Registry
HighLegal·2026·Binnall Law Group PLLC

Binnall Law Claude Console Phantom Citations in Federal Court

Federal counsel used Anthropic Claude Console to draft a May 2026 motion that included quotations not found in the cited cases. The firm apologised to the court and committed to citation verification via established legal research platforms.

D2 · Output ValidationD5 · Deployment SafetyD6 · Human Oversight

What happened

In AFGE v. Trump (N.D. Cal. No. 3:25-cv-03698), Binnall Law Group counsel used Claude Console under deadline pressure to draft a motion to quash a subpoena. An associate reviewed the draft but quoted language was not verified against the cited opinions. Judge Susan Illston flagged phantom quotations; partners filed apologies and described policy gaps, committing to supplemental training and Westlaw/Lexis citation checks.

PSF Analysis

How the Production Safety Framework maps to this failure

A D2 + D6 failure in a high-consequence legal workflow, compounded by D5 gaps: generative drafting under deadline without a fileable-path gate. The episode extends the court-filing hallucination pattern to a named enterprise console in active federal litigation.

Controls that would have prevented this

Specific PSF controls mapped to each failure point

1
D2 · Output Validation
Match every quoted string against retrieved opinion text before any filing export.
2
D6 · Human Oversight
Named accountable reviewer with citation-integrity checklist before e-filing.
3
D5 · Deployment Safety
Deployment gate — no generative draft enters court filing without a completed citation attestation record.

Outcome

Public apologies to the court; firm policy and training updates reported May 2026. Widely cited alongside the broader wave of generative-AI citation sanctions in litigation.

legalhallucinationcourt-filingscitation-validationclaude-consolefederal-court

Related incidents

High2024
Air Canada Chatbot Bereavement Fare
D1D5
High2025
Law Firm AI Hallucinated Fake Case Law in Court Filings
D2D6
High2026
OpenAI Multi-Service Outage (ChatGPT, Login, Access)
D8D4D5
NEXT STEP

Map this failure back to the standard

Use the PSF domains behind this incident to define review gates, remediation evidence, and safer production requirements.

Read the PSF →← All incidents