New AI Exam Fallout: California Bar’s Tech Blunder Sparks Outcry

The State Bar of California has admitted that it used artificial intelligence—and even recycled student exam questions—to create parts of its February bar exam. What was meant to cut costs and speed up test development has instead ignited fierce criticism from law schools, test-takers, and legislators alike.

AI-Powered Questions: A Cost-Cutting Gamble

Facing a $22 million deficit, the Bar ditched the standard multistate exam and paid vendors to build a hybrid online-in-person test. Of the 171 scored multiple-choice items, 100 came from Kaplan Exam Services, 48 were lifted from a first-year law student test, and 23 were drafted by ACS Ventures using AI. Although Bar officials insist all questions passed legal-accuracy reviews and met psychometric targets, the rush to employ non-lawyer AI assistance rattled confidence in the exam’s fairness.

Law Schools and Experts Sound the Alarm

Academic experts argue that only seasoned jurists can craft questions that test not just knowledge but real-world application. “Having non-lawyers draft questions with AI is unbelievable,” said one assistant dean. Critics also raised conflict-of-interest concerns: the same psychometrician who used AI to write questions then validated them. Meanwhile, glitches in the online platform—from screen freezes to erroneous prompts—left many test-takers frustrated and considering legal action.

Fixing the Flaws: What’s Next?

The Bar has asked the California Supreme Court to adjust scores for affected candidates and will review non-scoring questions. A court spokesperson said justices only learned of the AI use this week. Meanwhile, legislators are calling for audits and tighter oversight, and the July exam is set to revert to in-person testing. The Committee of Bar Examiners meets May 5 to decide on remedies—but transparency advocates want full release of all 200 questions before any future AI drafting.

Frequently Asked Questions

Q1: How was AI used on the bar exam?
A subset of multiple-choice questions—23 out of 171—were drafted by ACS Ventures using AI tools, while another 48 were directly reused from a first-year law exam.

Q2: Why is this controversial?
Law educators say only experienced legal professionals can frame questions that truly assess competence. Using AI and non-lawyers raised concerns about question quality, bias, and conflict of interest in validation.

Q3: What relief is being offered to test-takers?
The State Bar has requested the California Supreme Court adjust scores for those affected and will shift the July exam back to fully in-person administration with traditional question development.

Sources Los Angeles Times