Sullivan & Cromwell is one of the most respected law firms in the country. They advise OpenAI on “safe and ethical deployment” of artificial intelligence. They have two mandatory AI training modules with tracked completions. Their office manual tells lawyers to “trust nothing and verify everything.” They have policies. They have safeguards.

Last week, a partner at the firm wrote an emergency letter to a federal bankruptcy judge admitting that their filings were riddled with AI hallucinations.

What happened

S&C filed an emergency motion in the Chapter 15 bankruptcy of Prince Global Holdings in the Southern District of New York. After the filing, the firm discovered that the motion, the verified petition, scheduling motions, and supporting declarations contained fabricated citations, wrong volume numbers, incorrect pin cites, and parenthetical quotes that simply do not appear in the cited cases.

The firm’s letter to Chief Bankruptcy Judge Martin Glenn catalogued roughly 40 corrections across multiple filings. The partner’s letter was blunt about what went wrong:

“The Firm’s policies on the use of AI were not followed in connection with the preparation of the Motion.”

That one sentence is the whole story. The policies existed. The training existed. The completion tracking existed. None of it prevented the error because none of it forced verification at the point where it mattered.

Why this is different from Mata v. Avianca

When Steven Schwartz submitted fake ChatGPT-generated citations in 2023, the reaction was mostly “he should have known better.” A solo practitioner who didn’t understand the tool. Easy to dismiss.

This is not that.

Sullivan & Cromwell is an Am Law 10 firm with a dedicated AI program, mandatory training, and tracked completions. Their office manual has language specifically about verifying AI output. And they still filed hallucinated citations with a federal court. The problem was not ignorance. The verification step between “AI generated this” and “we filed it” broke down despite every safeguard they had on paper.

As Above the Law noted, this is not a Sullivan & Cromwell problem. This is a profession problem. Researcher Damien Charlotin has now catalogued over 1,000 AI hallucination cases in legal filings worldwide. The numbers stopped being surprising a while ago. What’s new here is the proof that comprehensive policies don’t solve it.

The real lesson: policies decay without enforcement

Here’s what I think is actually going on. When AI output was obviously rough, lawyers reviewed it carefully. As the tools improved and started producing work that looked polished, the review got lighter. Nobody decided to skip verification. The output just stopped triggering the instinct to double-check.

S&C’s letter essentially describes this: the policies say verify everything, the manual says verify everything, the training says verify everything. But somewhere between the AI output and the filing, the actual verification didn’t happen.

This is a predictable failure mode. Policies tell people what to do. They don’t make people do it. And as AI output gets better, the gap between “what the policy says” and “what actually happens” widens. If your firm has an AI acceptable use policy, that’s a starting point, not a finish line.

What this means for smaller firms

If Sullivan & Cromwell, with their resources and their dedicated AI program, couldn’t prevent this with policies alone, a 30-person firm relying on a one-page AI policy and an annual training session is in a worse position. Smaller firms aren’t less capable, but they have fewer layers of review and less infrastructure to catch errors before they reach a court.

A few things worth thinking about:

Your policy should not just say “verify AI output.” It should specify who verifies, when in the process, and what verification actually looks like. Print the cited cases. Check the pin cites. Read the parenthetical quotes against the actual opinions. If that sounds tedious and expensive, it is. That’s the cost of using AI for legal work.

Products like BriefCatch’s RealityCheck are designed specifically to catch hallucinated citations in legal briefs. If your firm is using AI for research or drafting, a second layer of automated verification is worth the investment.

If you’re already disclosing AI use to clients, S&C’s incident is a reminder that disclosure alone isn’t enough. Clients are going to start asking what your verification process looks like, not just whether you use AI.

And expect this to get harder. AI tools are becoming more autonomous, taking actions inside documents rather than just suggesting edits. The more the tool does on its own, the more deliberate your review process needs to be.

The bottom line

Having an AI policy is necessary. It is not sufficient. Sullivan & Cromwell had everything a reasonable firm could have: policies, training, tracked completions, and a culture that explicitly told lawyers to verify AI output. The verification still didn’t happen.

The firms that avoid becoming the next headline will be the ones that treat verification as a workflow requirement, not a cultural expectation. Someone has to check the work. Every time. And that process has to be built into how the work gets done, not just written into a manual that sits on a shelf.