Catalin Voss

Catalin Voss

Auto-generated profile, based on publicly available information

Catalin Voss

Department of Computer Science, Stanford University, Stanford, CA, United States

Research Focus & Metrics

Academic Career Length

8Years (2019-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
22 April 2020 in Journal of Medical Internet Research
Verified
Journal Article
Open access
ANAnish Nag
Anish Nag
NHNick Haber
Nick Haber
CVCatalin Voss
Catalin Voss
STSerena Tamura
Serena Tamura
JDJena Daniels
Jena Daniels
DWDennis Wall
Dennis Wall
Background: Several studies have shown that facial attention differs in children with autism. Measuring eye gaze and emotion recognition in children with autism is challenging, as standard clinical assessments must be delivered in clinical settings by a trained clinician. Wearable technologies may be able to bring eye gaze and emotion recognition into natural social interactions and settings. Objective: This study aimed to test: (1) the feasibility of tracking gaze using wearable smart glasses during a facial expression recognition task and (2) the ability of these gaze-tracking data, together with facial expression recognition responses, to distinguish children with autism from neurotypical controls (NCs). Methods: We compared the eye gaze and emotion recognition patterns of 16 children with autism spectrum disorder (ASD) and 17 children without ASD via wearable smart glasses fitted with a custom eye tracker. Children identified static facial expressions of images presented on a computer screen along with nonsocial distractors while wearing Google Glass and the eye tracker. Faces were presented in three trials, during one of which children received feedback in the form of the correct classification. We employed hybrid human-labeling and computer vision–enabled methods for pupil tracking and world–gaze translation calibration. We analyzed the impact of gaze and emotion recognition features in a prediction task aiming to distinguish children with ASD from NC participants. Results: Gaze and emotion recognition patterns enabled the training of a classifier that distinguished ASD and NC groups. However, it was unable to significantly outperform other classifiers that used only age and gender features, suggesting that further work is necessary to disentangle these effects. Conclusions: Although wearable smart glasses show promise in identifying subtle differences in gaze tracking and emotion recognition patterns in children with and without ASD, the present form factor and data do not allow for these differences to be reliably exploited by machine learning systems. Resolving these challenges will be an important step toward continuous tracking of the ASD phenotype.

8 April 2022 in JMIR Pediatrics and Parenting
Verified
Journal Article
Open access
Peter WashingtonPeter Washington
Peter Washington
HKHaik Kalantarian
Haik Kalantarian
JKJohn Kent
John Kent
AHArman Husic
Arman Husic
AKAaron Kline
Aaron Kline
DWDennis Wall
Dennis Wall
Background: Automated emotion classification could aid those who struggle to recognize emotions, including children with developmental behavioral conditions such as autism. However, most computer vision emotion recognition models are trained on adult emotion and therefore underperform when applied to child faces. Objective: We designed a strategy to gamify the collection and labeling of child emotion–enriched images to boost the performance of automatic child emotion recognition models to a level closer to what will be needed for digital health care approaches. Methods: We leveraged our prototype therapeutic smartphone game, GuessWhat, which was designed in large part for children with developmental and behavioral conditions, to gamify the secure collection of video data of children expressing a variety of emotions prompted by the game. Independently, we created a secure web interface to gamify the human labeling effort, called HollywoodSquares, tailored for use by any qualified labeler. We gathered and labeled 2155 videos, 39,968 emotion frames, and 106,001 labels on all images. With this drastically expanded pediatric emotion–centric database (>30 times larger than existing public pediatric emotion data sets), we trained a convolutional neural network (CNN) computer vision classifier of happy, sad, surprised, fearful, angry, disgust, and neutral expressions evoked by children. Results: The classifier achieved a 66.9% balanced accuracy and 67.4% F1-score on the entirety of the Child Affective Facial Expression (CAFE) as well as a 79.1% balanced accuracy and 78% F1-score on CAFE Subset A, a subset containing at least 60% human agreement on emotions labels. This performance is at least 10% higher than all previously developed classifiers evaluated against CAFE, the best of which reached a 56% balanced accuracy even when combining “anger” and “disgust” into a single class. Conclusions: This work validates that mobile games designed for pediatric therapies can generate high volumes of domain-relevant data sets to train state-of-the-art classifiers to perform tasks helpful to precision health efforts.

23 May 2019 in Journal of Medical Internet Research
Verified
Journal Article
Open access
Peter WashingtonPeter Washington
Peter Washington
HKHaik Kalantarian
Haik Kalantarian
QTQandeel Tariq
Qandeel Tariq
JSJessey Schwartz
Jessey Schwartz
KDKaitlyn Dunlap
Kaitlyn Dunlap
DWDennis Wall
Dennis Wall
Journal of Medical Internet Research - International Scientific Journal for Medical Research, Information and Communication on the Internet #Preprint #PeerReviewMe: Warning: This is a unreviewed preprint. Readers are warned that the document has not been peer-reviewed by expert/patient reviewers or an academic editor, may contain misleading claims, and is likely to undergo changes before final publication, if accepted, or may have been rejected/withdrawn. Readers with interest and expertise are encouraged to sign up as peer-reviewer, if the paper is within an open peer-review period. Please cite this preprint only for review purposes or for grant applications and CVs (if you are the author). Background: Obtaining a diagnosis of neuropsychiatric disorders such as autism requires long waiting times often exceeding a year. Furthermore, obtaining a professional diagnosis is often prohibitively expensive for much of the global population. Crowdsourcing provides a cheap and possibly free alternative that can allow underserved populations to obtain an accurate diagnosis. Objective: To determine whether paid crowd workers on Amazon Mechanical Turk (AMT) as well as citizen crowd workers on a public website shared on social media can provide accurate online screening for autism from watching a short video clip. Methods: Three online studies: (1) a paid crowdsourcing task on AMT (N=55) where crowd workers were asked to rate ten short video clips of children as having “autism” or “no autism”, (2) a follow-up, more complex paid crowdsourcing task on AMT (N=27) with only those raters who correctly rated 8 or more of the 10 videos during the first study, and (3) a public unpaid “citizen healthcare” study (N=71) with the same task as in the first study. 50% of the videos in all studies contained children with autism, 50% of videos contained females, and the age range of children in the videos ranged from 2 to 5 years. Results: Study 1: The mean score of the participants who completed all questions was 7.50 (SD = 1.46) out of 10. When only analyzing the workers who scored reasonably well (scored 8 out of 10 or higher; n=27 out of 55), there was a weak negative correlation between the time spent rating the videos and the sensitivity (r = -0.44, p = 0.02). Study 2: The mean score of the participants rating new videos was 6.76 (SD = 0.59) out 10. Across all videos, the average deviation between the average crowdsourced answers and gold standard ratings provided by two expert clinical research coordinators was 0.557, with a standard deviation of 0.51 (the maximum possible deviation is 3). All paid crowdworkers who scored an 8 out of 10 in Study 1 either enjoyed performing the task in Study 2 or provided no negative comments. Study 3: There was a negative correlation between whether the rater is a parent and sensitivity (r = -0.26, p=0.02) as well as precision (r = -0.28, p=0.02). The mean score of the participants who completed all questions was 6.61 (SD = 1.60) out of 10. A two-tailed t-test between the scores of the paid workers in Study 1 and the unpaid workers in Study 3 showed significance in the difference (p=0.0009). Conclusions: (1) Many paid crowdworkers on AMT enjoy answering screening questions from videos, suggesting higher intrinsic motivation to make quality assessments.”). (2) Both paid crowdsourcing and citizen crowdsourcing provide promising screening assessments of pediatric autism comparable to that of professional raters. (3) Parents of children with autism likely overfit their intuition to their own affected child, which is problematic for a disorder such as autism which has a wide range of phenotypic manifestations. (4) Paid workers will outperform a general unpaid crowd of online citizen workers.

23 December 2020 in JMIR Preprints
Verified
Preprint
Open access
Peter WashingtonPeter Washington
Peter Washington
HKHaik Kalantarian
Haik Kalantarian
JKJohn Kent
John Kent
AHArman Husic
Arman Husic
AKAaron Kline
Aaron Kline
DWDennis Wall
Dennis Wall
BACKGROUND Automated emotion classification could aid those who struggle to recognize emotions, including children with developmental behavioral conditions such as autism. However, most computer vision emotion recognition models are trained on adult emotion and therefore underperform when applied to child faces. OBJECTIVE We designed a strategy to gamify the collection and labeling of child emotion–enriched images to boost the performance of automatic child emotion recognition models to a level closer to what will be needed for digital health care approaches. METHODS We leveraged our prototype therapeutic smartphone game, GuessWhat, which was designed in large part for children with developmental and behavioral conditions, to gamify the secure collection of video data of children expressing a variety of emotions prompted by the game. Independently, we created a secure web interface to gamify the human labeling effort, called HollywoodSquares, tailored for use by any qualified labeler. We gathered and labeled 2155 videos, 39,968 emotion frames, and 106,001 labels on all images. With this drastically expanded pediatric emotion–centric database (>30 times larger than existing public pediatric emotion data sets), we trained a convolutional neural network (CNN) computer vision classifier of happy, sad, surprised, fearful, angry, disgust, and neutral expressions evoked by children. RESULTS The classifier achieved a 66.9% balanced accuracy and 67.4% F1-score on the entirety of the Child Affective Facial Expression (CAFE) as well as a 79.1% balanced accuracy and 78% F1-score on CAFE Subset A, a subset containing at least 60% human agreement on emotions labels. This performance is at least 10% higher than all previously developed classifiers evaluated against CAFE, the best of which reached a 56% balanced accuracy even when combining “anger” and “disgust” into a single class. CONCLUSIONS This work validates that mobile games designed for pediatric therapies can generate high volumes of domain-relevant data sets to train state-of-the-art classifiers to perform tasks helpful to precision health efforts.

27 June 2019 in Journal of Medical Internet Research
Verified
Journal Article
Open access
Peter WashingtonPeter Washington
Peter Washington
HKHaik Kalantarian
Haik Kalantarian
QTQandeel Tariq
Qandeel Tariq
JSJessey Schwartz
Jessey Schwartz
KDKaitlyn Dunlap
Kaitlyn Dunlap
DWDennis Wall
Dennis Wall
Journal of Medical Internet Research - International Scientific Journal for Medical Research, Information and Communication on the Internet

9 February 2019 in JMIR Preprints
Verified
Preprint
Open access
Peter WashingtonPeter Washington
Peter Washington
HKHaik Kalantarian
Haik Kalantarian
QTQandeel Tariq
Qandeel Tariq
JSJessey Schwartz
Jessey Schwartz
KDKaitlyn Dunlap
Kaitlyn Dunlap
DWDennis Wall
Dennis Wall
BACKGROUND Obtaining a diagnosis of neuropsychiatric disorders such as autism requires long waiting times that can exceed a year and can be prohibitively expensive. Crowdsourcing approaches may provide a scalable alternative that can accelerate general access to care and permit underserved populations to obtain an accurate diagnosis. OBJECTIVE We aimed to perform a series of studies to explore whether paid crowd workers on Amazon Mechanical Turk (AMT) and citizen crowd workers on a public website shared on social media can provide accurate online detection of autism, conducted via crowdsourced ratings of short home video clips. METHODS Three online studies were performed: (1) a paid crowdsourcing task on AMT (N=54) where crowd workers were asked to classify 10 short video clips of children as “Autism” or “Not autism,” (2) a more complex paid crowdsourcing task (N=27) with only those raters who correctly rated ≥8 of the 10 videos during the first study, and (3) a public unpaid study (N=115) identical to the first study. RESULTS For Study 1, the mean score of the participants who completed all questions was 7.50/10 (SD 1.46). When only analyzing the workers who scored ≥8/10 (n=27/54), there was a weak negative correlation between the time spent rating the videos and the sensitivity (ρ=–0.44, P=.02). For Study 2, the mean score of the participants rating new videos was 6.76/10 (SD 0.59). The average deviation between the crowdsourced answers and gold standard ratings provided by two expert clinical research coordinators was 0.56, with an SD of 0.51 (maximum possible SD is 3). All paid crowd workers who scored 8/10 in Study 1 either expressed enjoyment in performing the task in Study 2 or provided no negative comments. For Study 3, the mean score of the participants who completed all questions was 6.67/10 (SD 1.61). There were weak correlations between age and score (r=0.22, P=.014), age and sensitivity (r=–0.19, P=.04), number of family members with autism and sensitivity (r=–0.195, P=.04), and number of family members with autism and precision (r=–0.203, P=.03). A two-tailed t test between the scores of the paid workers in Study 1 and the unpaid workers in Study 3 showed a significant difference (P<.001). CONCLUSIONS Many paid crowd workers on AMT enjoyed answering screening questions from videos, suggesting higher intrinsic motivation to make quality assessments. Paid crowdsourcing provides promising screening assessments of pediatric autism with an average deviation <20% from professional gold standard raters, which is potentially a clinically informative estimate for parents. Parents of children with autism likely overfit their intuition to their own affected child. This work provides preliminary demographic data on raters who may have higher ability to recognize and measure features of autism across its wide range of phenotypic manifestations.