Published on December 19, 2023 by Sofia Paglioni  
female business professor DR03072023137

New Artificial Intelligence (AI) tools made a noticeable entrance into many industries in 2023, including education. Some educators are skeptical about embracing this new technology, however both students and professional writers are seeking to utilize AI.

Recently, three faculty members in Samford University’s Brock School of Business, Joy Buchanan, Stephen Hill and Olga Shapoval, created a novel data set relevant to AI writing. Their paper “ChatGPT Hallucinates Non-existent Citations: Evidence from Economics” has been accepted for publication in the journal The American Economist. This past summer, it was featured in the Wall Street Journal.

male business professor DR080120231189In the article, they provide quantitative evidence for errors in ChatGPT output to demonstrate the importance of LLM verification. When they prompted ChatGPT to write citations, the software would create false or non-existent references.

Buchanan commented on her research and said, “I expect that this specific problem with the fake citations will be solved. It’s very brazen. When people understand this problem, they are shocked. Fake citations? It printed out reference for papers that do not actually exist? Yes, it really did that. These errors are easy to catch, so I expect ChatGPT will clean up its own mess on this particular issue quickly. However, that does not mean that the more general issue of hallucinations is going away. This paper will serve as bright clear evidence that GPT can hallucinate in ways that detract from the quality of the output or even pose safety concerns in some use cases. This generalizes far beyond academic citations.”

female business professor DR09122023697

In a follow-up study with William Hickman, Buchanan is further investigating the issue of writing with AI. ChatGPT can be very useful, but human readers should not trust every fact that it reports. Buchanan and Hickman study whether people will pay for a fact-check on AI writing. Participants who are not explicitly informed of authorship tend to trust statements they believe are human-written more than those attributed to ChatGPT. However, there are several unexpected results that you can read about in their working paper Do People Trust Humans More Than ChatGPT?. This paper has been featured on the widely-read economics blog Marginal Revolution and it is among the most-downloaded papers of the month on SSRN, an academic research repository.
Samford is a leading Christian university offering undergraduate programs grounded in the liberal arts with an array of nationally recognized graduate and professional schools. Founded in 1841, Samford is the 87th-oldest institution of higher learning in the United States. Samford enrolls 5,791 students from 49 states, Puerto Rico and 16 countries in its 10 academic schools: arts, arts and sciences, business, divinity, education, health professions, law, nursing, pharmacy and public health. Samford fields 17 athletic teams that compete in the tradition-rich Southern Conference and ranks 6th nationally for its Graduation Success Rate among all NCAA Division I schools.