University of Minnesota law professors test if AI program ChatGPT can pass their final exams

“The questions we ask on law school exams are not ‘tell me when this law was passed’, or the types of things you can ask Google to give you an answer,” Schwarcz said.

In the end, the test taken by ChatGPT passed with a C+

“It’s a passing grade at a really good law school,” said Schwarcz. “Our students are really excellent. To get a passing grade, it’s very difficult questions.”

Students at the U are chatting about it too.

“It’s all over Twitter, it’s all over social media,” said student Brian Gatter. “The part that almost surprised me is that it only got a C+.”

Schwarcz says now it’s on teachers to make sure it’s not used to cheat, but rather think of new ways it can be used for good.

“AI can’t just write a brief, it can’t just write a will. But what it can do is be a useful tool,” he said.

Schwarcz says in the long run, AI likely won’t end up replacing attorneys, but helping them. 

The Research is presented in a 16-page report called “ChatGPT goes to Law school.” According to the report, despite the C+ grade, the AI exceeded in some areas of the test while struggling in others. Ninety-nine percent of students who pass that test pass the bar.