(Image credit: Dall E)

For fans of real-life legal drama, I’ve got a doozy for you. An attorney in a New York Supreme Court legal case was accused by a plaintiff and their lawyers of providing “inaccurate citations and quotations” that appeared to be “hallucinated” by an AI tool.

When said lawyer opposed the claims, they were then accused of defending their use of AI with, well, AI (via 404 Media). Or, to put it more succinctly, in the words of judge Joel Coehn:

“Counsel relied upon unvetted AI—in his telling, via inadequately supervised colleagues—to defend his use of unvetted AI.”

The case itself involved a dispute between family members and a defaulted loan, usually a relatively straightforward proceeding. However, once the plaintiff brought to the court’s attention the apparently AI-generated citations and quotes in the defense counsel’s brief, the defense claimed that “these passages were intended as paraphrases or summarized statements of the legal principles established in the cited authorities.”

The judge points out that the purported paraphrases included “bracketed terms to indicate departure from a quotation (not something one would expect to see in an intended paraphrase) and comments such as ‘citation omitted.’ Moreover, the cited cases often did not stand for the propositions quoted, were completely unrelated in subject matter, and in one instance did not exist at all.”

(Image credit: Capcom)

After being called out on the potential use of AI by the plaintiff, the defense then provided opposition to the claim, which was also later deemed to be made with the help of AI. According to the judge:

“This time, [the] plaintiff has identified more than double the number of mis-cites, including four citations that do not exist, seven quotations that do not exist in the cited cases, and three that do not support the propositions for which they are offered.”

Keep up to date with the most important stories and the best deals, as picked by the PC Gamer team.

The judge called the case “yet another unfortunate chapter to the story of artificial intelligence misuse in the legal profession,” and stated that, while some of the citations happened to contain “arguably correct statements of law”, it made them “no less frivolous.”

Well, that must have made for an awkward courtroom. Still, as the judge references, this is far from the first time the use of AI has reared its ugly head in legal proceedings. Earlier this year, a judge ordered a ChatGPT-using lawyer to not only pay $5,500 in sanctions, but to attend a plenary session entitled “Smarter Than Ever: The Potential and Perils of Artificial Intelligence” for their transgressions.

It appears that courts look unfavourably upon AI avatars for personal representation, too. Back in April, a plaintiff in an employment dispute attempted to use just such an AI stand-in to deliver their opening presentation, as they felt their tendency to mumble would negatively affect their case, only for the artificial representative to be shut down by the judge before it could finish its first sentence.

(Image credit: Getty Images / Brian A. Jackson/South Florida Sun Sentinel)

All of this comes as no surprise, I reckon. The legal profession is a complicated business, and given that AI tools are often touted for their ability to scythe through dense information to pull out the relevant facts, it’s no great wonder that some are taking AI-based shortcuts in legal proceedings.

And so far, they seem to usually come unstuck. However, part of me wonders how much AI-generated content has already made its way into legal cases without being noticed. In this case, the plaintiff and their counsel appear to have been on the ball, and as a result the judge was able to call out the discrepancy.

My favourite law, though, the law of averages, suggests that at least some AI-generated arguments, citations, and quotations might have made it through the cracks. Pessimistic? Me? Never. I’m just a happy-go-lucky bundle of joy.

Andy built his first gaming PC at the tender age of 12, when IDE cables were a thing and high resolution wasn’t—and he hasn’t stopped since. Now working as a hardware writer for PC Gamer, Andy spends his time jumping around the world attending product launches and trade shows, all the while reviewing every bit of PC gaming hardware he can get his hands on. You name it, if it’s interesting hardware he’ll write words about it, with *opinions *and everything.