GPT-4, the newest chatbot from OpenAI, has gained attention for its remarkable language ability. One recent study, however, looked at how the AI would perform on accounting examinations, and the findings were startling.
David Wood, a professor of accounting at BYU, led the study, which included 25,181 classroom accounting exam questions from 327 co-authors from 186 educational institutions across 14 countries. Additionally, they enlisted BYU undergrads to provide ChatGPT with 2,268 additional textbook test bank questions. The questions, which ranged in complexity and type, were about accounting information systems (AIS), auditing, financial accounting, managerial accounting, and tax.
Although ChatGPT performed well, earning a score of 47.4%, the students outperformed it, earning an average score of 76.7%. Although ChatGPT struggled with tax, financial, and managerial assessments, it performed particularly well on AIS and auditing. This may be because ChatGPT had trouble with the mathematical processes needed for the latter type of assessment.
Highlight:
- The outstanding language abilities of GPT-4, the newest chatbot from OpenAI, have garnered media attention.
- In a recent study, it was determined to see how well the AI would perform on accounting tests.
- With a score of 47.4%, ChatGPT performed admirably, but the students outperformed them with an average score of 76.7%.
- Additionally, the researchers discovered that ChatGPT frequently offers justifications for incorrect responses, struggles to recognize when it is performing math, and occasionally invents facts.
- Despite its flaws, ChatGPT has the potential to enhance teaching and learning, including the capacity to design and test assignments or rough drafts of projects, according to the study’s authors.
- Overall, the study’s findings show that, despite ChatGPT’s flaws and need for improvement in the area of accounting, AI technology has the ability to transform education and enhance the teaching and learning experience for all parties involved.
Additionally, the researchers discovered that ChatGPT frequently offers justifications for incorrect responses, occasionally invents facts, and doesn’t always recognize when it is performing math. They do, however, anticipate that the shortcomings described above will be vastly improved by GPT-4.
The chatbot, despite its flaws, was determined to be a game-changer that will alter how everyone teaches and learns by the study’s authors. They recognize ChatGPT’s potential to enhance teaching and learning, including the capacity to create and test assignments or rough drafts of projects.
“This is a disruption, and we need to assess where we go from here,” said study coauthor and fellow BYU accounting professor Melissa Larson. “Of course, I’m still going to have TAs, but this is going to force us to use them in different ways.”
Overall, the study’s findings demonstrate that while AI technology like ChatGPT is not perfect and still has work to do in the realm of accounting, it has the potential to revolutionize education and improve the teaching and learning process for everyone involved.
Content Writing
Video Marketing
Graphic Design
Lead Magnet Creation
Content Marketing