It seems like every few weeks, another headline screams about a lawyer facing consequences for submitting court filings riddled with AI-generated inaccuracies. Judges are slamming these filings as containing “bogus AI-generated research,” and the details vary, but the core issue remains the same: lawyers are using large language models (LLMs) like ChatGPT for legal research or even writing, and these LLMs are “hallucinating” non-existent cases. This results in lawyers unknowingly submitting false information, leading to sanctions and fines.
So, why do lawyers persist in using these tools despite the obvious pitfalls? The answer largely boils down to time constraints and the pervasive integration of AI across various professions. Legal research platforms like LexisNexis and Westlaw already incorporate AI, making it seem like an incredibly efficient assistant for lawyers managing heavy workloads. While many aren’t using ChatGPT for drafting filings, its use for research is growing. However, many lawyers, mirroring the public’s understanding, lack a full grasp of LLMs and their functionalities. One sanctioned attorney even believed ChatGPT was merely a “super search engine,” a misconception that led to a filing filled with fabricated citations.
Andrew Perlman, dean of Suffolk University Law School, points out that many lawyers use AI tools without incident, viewing the cases of fabricated citations as outliers. He emphasizes the potential benefits and various use cases for AI in legal services. A 2024 Thomson Reuters survey supports this, revealing that 63% of lawyers have used AI, with 12% using it regularly. Respondents cited time savings as a key benefit, primarily using AI for summarizing case law and researching legal precedents.
However, the reality is that AI-generated documents aren’t always accurate, and sometimes completely fabricated. A high-profile case involving journalist Tim Burke highlights this risk. His lawyers, using ChatGPT’s “deep research” feature and Westlaw’s AI capabilities, submitted a motion containing nine hallucinations, as identified by Judge Kathryn Kimball Mizelle. Similar incidents involving Anthropic’s Claude AI and misinformation expert Jeff Hancock further underscore this issue.
These inaccuracies significantly impact court proceedings. A California judge, initially persuaded by arguments in a brief, later discovered the cited case law was entirely fictitious. This highlights the crucial role of accuracy in legal documents. Perlman suggests less risky applications of generative AI, such as analyzing large datasets of discovery documents or brainstorming arguments. He stresses that AI should supplement, not replace, legal expertise and judgment.
Despite the risks, lawyers continue to use AI tools, often due to time pressures. Even before LLMs, lawyers sometimes submitted documents with inaccurate citations, but the nature of the errors has changed. The seductive ease of AI-generated text can also lead to over-reliance and a lack of sufficient verification. Alexander Kolodin, an Arizona state representative and election lawyer, uses ChatGPT as a “junior-level associate,” always verifying citations before submission. He even incorporated AI-generated text into a deepfake bill, highlighting the increasing integration of AI into the legal process.
The prevalence of AI in legal practice prompted the American Bar Association to issue guidance in 2024, emphasizing the duty of competence and the need for lawyers to understand the risks and benefits of AI tools. This includes assessing confidentiality risks and considering whether to disclose AI usage to clients.
While Perlman remains optimistic about AI’s transformative potential in the legal field, others, including judges who have sanctioned lawyers for AI-related errors, maintain a more cautious stance. The consensus is clear: while AI offers efficiency gains, thorough verification of AI-generated content remains crucial to maintain legal integrity and avoid future mishaps.
