US Law Professors Prefer AI Answers Over Human Ones in Study

In a study, 16 US law professors rated AI-generated answers better than human-written ones. This is a big change for legal schools.
By Newsroom | Education, Technology |

Law school professors involved in a recent study expressed a marked preference for answers generated by artificial intelligence (AI) over those penned by their peers. This preference held across various question types and evaluators, suggesting a consensus on the qualities of AI-generated responses. The study involved 16 contracts professors from 14 U.S. law schools, who collaborated to create 40 questions. These questions were then used to solicit answers, with a total of 2,918 anonymized comparisons between human and AI outputs being judged.

The research indicated a consistent advantage for AI-generated answers, regardless of the category of the question. These categories spanned factual recall (case or code, doctrine), hypothetical scenarios, and policy-related inquiries. The evaluations suggest that the LLM-generated answers met shared professional standards more effectively than human responses in this context.

This finding points to a potential shift in how legal knowledge is assessed and perhaps even disseminated. The study, as reported, involved participants creating questions and then judging the resulting answers. The structure of the comparison—anonymized and across various instructional domains—aims to mitigate bias.

Read More: Morning Word Usage Changes: Simple Meanings Explained

Further complicating the landscape, reports from institutions like Georgetown University highlight the existing imperfections and potential biases within AI systems themselves, often stemming from flawed training data. This introduces a tension between the perceived utility of AI and its inherent limitations.

The pressure to adopt AI in legal education is mounting. Articles emphasize that law graduates are increasingly expected to possess technological expertise, a skill gap that many law schools have been slow to address. Firms and legal departments are already leveraging AI for efficiency and cost reduction, potentially marginalizing new lawyers lacking AI familiarity.

Meanwhile, the development of AI tools for legal education is progressing, with platforms offering ways for faculty to create teaching prototypes. These tools aim to provide individualized skills practice, a persistent challenge for law professors managing large student cohorts. The creation of these custom AI tools is reportedly less technically demanding than commonly assumed, and designed with intellectual property protection in mind.

Read More: Digital Notes Used as Wills Cause Legal Problems in UK Courts

The impact of AI on the legal profession is framed as undeniable. As AI systems become more capable, they perform tasks historically handled by legal professionals. This technological evolution brings both benefits and ethical considerations, prompting questions about how AI is transforming daily legal work and what it signifies for practitioners today. The articles suggest a broader conversation is needed on how legal education must adapt to remain relevant in an AI-infused future, with some arguing that ignoring these advancements carries significant costs.

Frequently Asked Questions

Q: Why did 16 US law professors prefer AI answers in a recent study?
A study found that 16 contracts professors from 14 US law schools preferred AI-generated answers over human-written ones. They judged nearly 3,000 comparisons and found AI answers met professional standards better.
Q: Did AI answers perform better than human answers for all types of legal questions?
Yes, the study showed AI had an advantage for all question types. This included factual recall, hypothetical scenarios, and policy questions, suggesting AI is seen as more consistent.
Q: How might this preference for AI answers affect law schools and future lawyers?
This could lead to changes in how legal knowledge is taught and tested. Law graduates may need more AI skills as firms already use AI for efficiency, potentially leaving those without AI knowledge behind.
Q: Are there concerns about using AI in legal education?
Yes, there are concerns. AI systems can have biases or imperfections from their training data. This creates a tension between AI's usefulness and its limitations, which law schools need to consider.
Q: Is it hard for law schools to create AI tools for teaching?
No, the study suggested that creating custom AI tools for teaching might be less difficult than people think. These tools can help give students individual practice, which is hard for professors with many students.