Xiner Liu

Xiner Liu

Auto-generated profile, based on publicly available information

Xiner Liu

University of Pennsylvania

Research Focus & Metrics

Academic Career Length

4Years (2023-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
17 March 2025 in Journal of Learning Analytics
Verified
Journal Article
Open access
PDF file
XLXiner Liu
Xiner Liu
Andres Felipe ZambranoAndres Felipe Zambrano
Andres Felipe Zambrano
RBRyan S. Baker
Ryan S. Baker
ABAmanda Barany
Amanda Barany
JOJaclyn Ocumpaugh
Jaclyn Ocumpaugh
ZWZhanlan Wei
Zhanlan Wei
This study explores the potential of the large language model GPT-4 as an automated tool for qualitative data analysis by educational researchers, exploring which techniques are most successful for different types of constructs. Specifically, we assess three different prompt engineering strategies — Zero-shot, Few-shot, and Few-shot with contextual information — as well as the use of embeddings. We do so in the context of qualitatively coding three distinct educational datasets: Algebra I semi-personalized tutoring session transcripts, student observations in a game-based learning environment, and debugging behaviours in an introductory programming course. We evaluated the performance of each approach based on its inter-rater agreement with human coders and explored how different methods vary in effectiveness depending on a construct’s degree of clarity, concreteness, objectivity, granularity, and specificity. Our findings suggest that while GPT-4 can code a broad range of constructs, no single method consistently outperforms the others, and the selection of a particular method should be tailored to the specific properties of the construct and context being analyzed. We also found that GPT-4 has the most difficulty with the same constructs than human coders find more difficult to reach inter-rater reliability on.

2 November 2024
Verified
Book Chapter
Closed access
PDF file
XLXiner Liu
Xiner Liu
JZJiayi Zhang
Jiayi Zhang
ABAmanda Barany
Amanda Barany
MPMaciej Pankiewicz
Maciej Pankiewicz
RBRyan S. Baker
Ryan S. Baker

6 October 2023
Verified
Conference Paper
Open access
XLXiner Liu
Xiner Liu
SSStefan Slater
Stefan Slater
JAJuliana Ma. Alexandra L. Andres
Juliana Ma. Alexandra L. Andres
LSLuke Swanson
Luke Swanson
JSJennifer Scianna
Jennifer Scianna
RBRyan S. Baker
Ryan S. Baker
The real-time detection of when a player is struggling presents an opportunity for game designers to design timely and meaningful interventions, as well as to provide targeted support that improves student learning and engagement. In this paper, we present a struggle detector in the context of students playing the learning game, Wake: Tales from the Aqualab. Using the interaction log data of the game, we engineered four sets of features that captured distinct aspects of gameplay and trained prediction models to identify human-coded cases of students struggling, cross-validating at the student level. Our best-performing detectors have shown some capability in identifying student struggles with modest performance, at an AUC (Area Under the Curve) value of 0.635. We discuss current limitations of this approach, as well as next steps towards providing real-time support within the game.

22 October 2023
Verified
Book Chapter
Closed access
PDF file
JSJennifer Scianna
Jennifer Scianna
XLXiner Liu
Xiner Liu
SSStefan Slater
Stefan Slater
RBRyan S. Baker
Ryan S. Baker


2 November 2024
Verified
Book Chapter
Closed access
PDF file
XLXiner Liu
Xiner Liu
AGAshish Gurung
Ashish Gurung
RBRyan S. Baker
Ryan S. Baker
ABAmanda Barany
Amanda Barany