H Human–AI Coevolution

Entry

Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools

Varun Magesh, Faiz Surani, Matthew Dahl, Mirac Suzgun, Christopher D. Manning, Daniel E. Ho

Synopsis

First preregistered empirical evaluation of commercial AI legal-research tools claiming to "eliminate" hallucinations. Lexis+ AI, Westlaw AI-Assisted Research, and Ask Practical Law AI each hallucinate 17–33% of the time. Quantifies the verification burden falling on attorneys when AI is sold as authoritative.

Keywords

·legal research ·retrieval-augmented generation ·hallucination ·verification burden ·preregistered evaluation

Open paper ↗ arXiv ↗ Report issue ↗