ChatGPT, a language-learning model chatbot, has garnered considerable attention for its ability to respond to users’ questions. Using data from 14 countries and 186 institutions, we compare ChatGPT and student performance for 28,085 questions from accounting assessments and textbook test banks. As of January 2023, ChatGPT provides correct answers for 56.5 percent of questions and partially correct answers for an additional 9.4 percent of questions. When considering point values for questions, students significantly outperform ChatGPT with a 76.7 percent average on assessments compared to 47.5 percent for ChatGPT if no partial credit is awarded and 56.5 percent if partial credit is awarded. Still, ChatGPT performs better than the student average for 15.8 percent of assessments when we include partial credit. We provide evidence of how ChatGPT performs on different question types, accounting topics, class levels, open/closed assessments, and test bank questions. We also discuss implications for accounting education and research.
18 January 2022 in Journal of International Financial Management & Accounting
SUMMARY: Audit data analytics (ADAs) allow auditors to analyze the entire population of transactions that has measurable benefits for audit quality. However, auditors caution that the level of assurance on the financial statements is not incrementally increased. We examine whether the testing methodology and the type of ICFR opinion issued affect jurors' perceptions of auditor negligence. We predict and find that when auditors issue an unqualified ICFR opinion, jurors make higher negligence assessments when auditors employ statistical sampling than when they employ ADAs. Further, when auditors issue an adverse ICFR opinion, jurors attribute less blame to auditors and more blame to the investor for an audit failure. Additionally, jurors perceive the use of ADAs as an indicator of higher audit quality and are less likely to find auditors negligent. However, jurors do not perceive a difference in the level of assurance provided when auditors use ADAs versus sampling testing methods.
Using Public Company Accounting Oversight Board (PCAOB) inspection data from 2006 to 2018, we examine the use of auditor‐employed specialists in audit engagements. First, we find that the use of specialists is increasingly prevalent and related to clients’ size and complex accounting estimates. Second, the use of specialists is positively associated with the incidence of audit process deficiencies (identified by PCAOB inspections) but is not associated with output‐based audit‐quality proxies (restatements or absolute discretionary accruals). Hence, although process deficiencies are more likely to occur in engagements with higher use of specialists, financial reporting quality is not negatively impacted. Third, the use of specialists is positively associated with the likelihood of goodwill impairments and negatively associated with engagement profitability. Finally, cross‐sectional tests suggest that board accounting expertise is a salient condition for more effective use of specialists. Collectively, our findings align with concerns noted by the PCAOB and prior experimental and survey studies. Although specialists assist auditors with the audit of complex estimates, engagements with comparatively high specialist use entail an incremental risk of audit process deficiencies.
1 September 2019 in Management Accounting Research
In October 2022, the International Auditing and Assurance Standards Board (IAASB) issued Exposure Draft 500 (ED‐500). This is focused on revising and integrating the standard auditors use when evaluating audit evidence during an external audit. This study contributes to the ongoing discourse as the IAASB evaluates feedback to ED‐500 and executes its standard‐setting agenda. We review academic literature published in the past 10 years to synthesize extant knowledge specifically on the use of technology and the application of professional skepticism during audit evidence evaluation. Our review offers factors the IAASB should consider when seeking to modernize and future‐proof its standards, suggesting improvements to the proposed ED‐500. We also identify fruitful avenues for future academic research.