Law Firm AI Hallucinated Fake Case Law in Court Filings

Reuters reported that major law firms were warning lawyers that AI can invent fake case law and that filing hallucinated citations can trigger sanctions. The issue moved from isolated embarrassment to a repeatable legal-risk pattern as generative AI use spread through litigation teams.

What happened

In early 2025, Reuters reported that Morgan & Morgan sent an urgent email to more than 1,000 lawyers warning that AI can invent case law and that any made-up authority placed into court papers could lead to sanctions. The wider legal industry had already seen multiple filings polluted by hallucinated citations. The underlying failure was not the use of AI itself, but the absence of mandatory citation verification before filing.

PSF Analysis

How the Production Safety Framework maps to this failure

This is a clean D2 + D6 failure. The model output was not validated against a source of truth before it entered a high-consequence legal process, and human oversight was treated as optional rather than mandatory. The broader lesson is that generative AI can assist legal drafting, but it cannot be allowed to invent authorities in a system where citations have legal effect. In PSF terms, the deployment path matters more than the model: if a tool can write into a court filing, it needs hard validation and human sign-off.

Controls that would have prevented this

Specific PSF controls mapped to each failure point

Outcome

Public warnings, sanctions risk, and tighter internal review policies across the sector. The case became a reference point for why legal drafting tools require citation-grounded output validation.

Related incidents

Map this failure back to the standard

Use the PSF domains behind this incident to define review gates, remediation evidence, and safer production requirements.