This research examines the performance of different artificial intelligence large language models (AI LLMs) on an Intermediate Accounting quiz focused on perpetual inventory, net method discounts, and FOB shipping terms. The goal is to evaluate each model's accuracy, speed, and self-reported confidence. We used an experimental approach in which 15 different AI LLM versions were presented with the identical multi-step scenario. A panel of accounting faculty validated the correct journal entries and scoring rubric to ensure uniformity between models.

Our results showed significant diversity in correctness (from 30% to 100%) and consistently high confidence (9 or 10 on a 1-10 scale). Misapplication of FOB destination rules, failure to properly record net method discounts, and accidental use of a periodic rather than perpetual system were among the most common errors. These findings highlight the importance of appropriate control when using AI in accounting. Instructors can promote deeper conceptual mastery by creating evaluations that challenge AI-driven shortcuts, whereas practitioners can pilot-test models, use quality-control checks, and be cautious as LLMs progress.

Authors:
Yunita Anwar, Shenandoah University, United States
Martin Mulyadi, Shenandoah University, United States

About the Presenter(s)
Dr. Yunita Anwar is the Lillian Cook Braun Endowed Chair in Accounting and an Assistant Professor of Accounting at Shenandoah University, Winchester, Virginia.

See this presentation on the full schedule – Sunday Schedule

Bookmark