Victoria Yaneva

Victoria Yaneva

Auto-generated profile, based on publicly available information

Victoria Yaneva

National Board of Medical Examiners (NBME)

Research Focus & Metrics

Academic Career Length

10Years (2017-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
30 April 2020 in IEEE Transactions on Neural Systems and Rehabilitation Engineering
Verified
Journal Article
Closed access
VYVictoria Yaneva
Victoria Yaneva
LHLe An Ha
Le An Ha
SESukru Eraslan
Sukru Eraslan
YYYeliz Yesilada
Yeliz Yesilada
RMRuslan Mitkov
Ruslan Mitkov
The purpose of this study is to test whether visual processing differences between adults with and without highfunctioning autism captured through eye tracking can be used to detect autism. We record the eye movements of adult participants with and without autism while they look for information within web pages. We then use the recorded eye-tracking data to train machine learning classifiers to detect the condition. The data was collected as part of two separate studies involving a total of 71 unique participants (31 with autism and 40 control), which enabled the evaluation of the approach on two separate groups of participants, using different stimuli and tasks. We explore the effects of a number of gaze-based and other variables, showing that autism can be detected automatically with around 74% accuracy. These results confirm that eye-tracking data can be used for the automatic detection of high-functioning autism in adults and that visual processing differences between the two groups exist when processing web pages.

7 November 2023 in Academic Medicine
Verified
Journal Article
Open access
VYVictoria Yaneva
Victoria Yaneva
PBPeter Baldwin
Peter Baldwin
DJDaniel P. Jurich
Daniel P. Jurich
KSKimberly Swygert
Kimberly Swygert
BCBrian E Clauser
Brian E Clauser
Purpose In late 2022 and early 2023, reports that ChatGPT could pass the United States Medical Licensing Examination (USMLE) generated considerable excitement, and media response suggested ChatGPT has credible medical knowledge. This report analyzes the extent to which an artificial intelligence (AI) agent’s performance on these sample items can generalize to performance on an actual USMLE examination and an illustration is given using ChatGPT. Method As with earlier investigations, analyses were based on publicly available USMLE sample items. Each item was submitted to ChatGPT (version 3.5) 3 times to evaluate stability. Responses were scored following rules that match operational practice, and a preliminary analysis explored the characteristics of items that ChatGPT answered correctly. The study was conducted between February and March 2023. Results For the full sample of items, ChatGPT scored above 60% correct except for one replication for Step 3. Response success varied across replications for 76 items (20%). There was a modest correspondence with item difficulty wherein ChatGPT was more likely to respond correctly to items found easier by examinees. ChatGPT performed significantly worse (P < .001) on items relating to practice-based learning. Conclusions Achieving 60% accuracy is an approximate indicator of meeting the passing standard, requiring statistical adjustments for comparison. Hence, this assessment can only suggest consistency with the passing standards for Steps 1 and 2 Clinical Knowledge, with further limitations in extrapolating this inference to Step 3. These limitations are due to variances in item difficulty and exclusion of the simulation component of Step 3 from the evaluation—limitations that would apply to any AI system evaluated on the Step 3 sample items. It is crucial to note that responses from large language models exhibit notable variations when faced with repeated inquiries, underscoring the need for expert validation to ensure their utility as a learning tool.

3 February 2020 in Universal Access in the Information Society
Verified
Journal Article
Closed access
PDF file
SESukru Eraslan
Sukru Eraslan
YYYeliz Yesilada
Yeliz Yesilada
VYVictoria Yaneva
Victoria Yaneva
LHLe An Ha
Le An Ha

5 May 2022 in Advances in Health Sciences Education
Verified
Journal Article
Open access
PDF file
VYVictoria Yaneva
Victoria Yaneva
BCBrian E. Clauser
Brian E. Clauser
AMAmy Morales
Amy Morales
MPMiguel Paniagua
Miguel Paniagua
Understanding the response process used by test takers when responding to multiple-choice questions (MCQs) is particularly important in evaluating the validity of score interpretations. Previous authors have recommended eye-tracking technology as a useful approach for collecting data on the processes test taker’s use to respond to test questions. This study proposes a new method for evaluating alternative score interpretations by using eye-tracking data and machine learning. We collect eye-tracking data from 26 students responding to clinical MCQs. Analysis is performed by providing 119 eye-tracking features as input for a machine-learning model aiming to classify correct and incorrect responses. The predictive power of various combinations of features within the model is evaluated to understand how different feature interactions contribute to the predictions. The emerging eye-movement patterns indicate that incorrect responses are associated with working from the options to the stem. By contrast, correct responses are associated with working from the stem to the options, spending more time on reading the problem carefully, and a more decisive selection of a response option. The results suggest that the behaviours associated with correct responses are aligned with the real-world model used for score interpretation, while those associated with incorrect responses are not. To the best of our knowledge, this is the first study to perform data-driven, machine-learning experiments with eye-tracking data for the purpose of evaluating score interpretation validity.

14 December 2021 in Journal of Educational Measurement
Verified
Journal Article
Open access
PDF file
VYVictoria Yaneva
Victoria Yaneva
BCBrian E. Clauser
Brian E. Clauser
AMAmy Morales
Amy Morales
MPMiguel Paniagua
Miguel Paniagua
Eye‐tracking technology can create a record of the location and duration of visual fixations as a test‐taker reads test questions. Although the cognitive process the test‐taker is using cannot be directly observed, eye‐tracking data can support inferences about these unobserved cognitive processes. This type of information has the potential to support improved test design and to contribute to an overall validity argument for the inferences and uses made based on test scores. Although several authors have referred to the potential usefulness of eye‐tracking data, there are relatively few published studies that provide examples of that use. In this paper, we report the results an eye‐tracking study designed to evaluate how the presence of the options in multiple‐choice questions impacts the way medical students responded to questions designed to evaluate clinical reasoning. Examples of the types of data that can be extracted are presented. We then discuss the implications of these results for evaluating the validity of inferences made based on the type of items used in this study.

24 February 2020 in Journal of Educational Measurement
Verified
Journal Article
Closed access
PDF file
PBPeter Baldwin
Peter Baldwin
VYVictoria Yaneva
Victoria Yaneva
JMJanet Mee
Janet Mee
BCBrian E. Clauser
Brian E. Clauser
LHLe An Ha
Le An Ha