New Artificial Intelligence (AI) tools made a noticeable entrance into many industries in 2023, including education. Some educators are skeptical about embracing this new technology, however both students and professional writers are seeking to utilize AI.
Recently, three faculty members in Samford University’s Brock School of Business, Joy Buchanan, Stephen Hill and Olga Shapoval, created a novel data set relevant to AI writing. Their paper “ChatGPT Hallucinates Non-existent Citations: Evidence from Economics” has been accepted for publication in the journal The American Economist. This past summer, it was featured in the Wall Street Journal.
In the article, they provide quantitative evidence for errors in ChatGPT output to demonstrate the importance of LLM verification. When they prompted ChatGPT to write citations, the software would create false or non-existent references.
Buchanan commented on her research and said, “I expect that this specific problem with the fake citations will be solved. It’s very brazen. When people understand this problem, they are shocked. Fake citations? It printed out reference for papers that do not actually exist? Yes, it really did that. These errors are easy to catch, so I expect ChatGPT will clean up its own mess on this particular issue quickly. However, that does not mean that the more general issue of hallucinations is going away. This paper will serve as bright clear evidence that GPT can hallucinate in ways that detract from the quality of the output or even pose safety concerns in some use cases. This generalizes far beyond academic citations.”
In a follow-up study with William Hickman, Buchanan is further investigating the issue of writing with AI. ChatGPT can be very useful, but human readers should not trust every fact that it reports. Buchanan and Hickman study whether people will pay for a fact-check on AI writing. Participants who are not explicitly informed of authorship tend to trust statements they believe are human-written more than those attributed to ChatGPT. However, there are several unexpected results that you can read about in their working paper Do People Trust Humans More Than ChatGPT?. This paper has been featured on the widely-read economics blog Marginal Revolution and it is among the most-downloaded papers of the month on SSRN, an academic research repository.