Xinyi Ai

Xinyi Ai

Auto-generated profile, based on publicly available information

Xinyi Ai

University of California, San Diego, San Diego, CA, USA

Research Focus & Metrics

Academic Career Length

3Years (2024-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
7 March 2024
Verified
Conference Paper
Open access
YSYiyin Shen
Yiyin Shen
XAXinyi Ai
Xinyi Ai
ARAdalbert Gerald Soosai Raj
Adalbert Gerald Soosai Raj
RJRogers Jeffrey Leo John
Rogers Jeffrey Leo John
MSMeenakshi Syamkumar
Meenakshi Syamkumar
ChatGPT is a conversational AI platform that can produce code to solve problems when provided with a natural language prompt. Prior work on similar AI models has shown that they perform well on typical intro-level Computer Science problems. However, little is known about the performance of such tools on Data Science (DS) problems. In this work, we assess the performance of ChatGPT on assignments from three DS courses with varying difficulty levels. First, we apply the raw assignment prompts provided to the students and find that ChatGPT performs well on assignments with dataset(s) descriptions and progressive question prompts, which divide the programming requirements into sub-problems. Then, we perform prompt engineering on the assignments for which ChatGPT had low performance. We find that the following prompt engineering techniques significantly increased ChatGPT's performance: breaking down abstract questions into steps, breaking down steps into multiple prompts, providing descriptions of the dataset(s), including algorithmic details, adding specific instructions to entice specific actions, and removing extraneous information. Finally, we discuss how our findings suggest potential changes to curriculum design of DS courses.