Essam Sleiman

Essam Sleiman

Auto-generated profile, based on publicly available information

Essam Sleiman

Harvard University

Research Focus & Metrics

Academic Career Length

5Years (2022-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
3 November 2024
Verified
Book Chapter
Closed access
PDF file
KAKirolos Ataallah
Kirolos Ataallah
XSXiaoqian Shen
Xiaoqian Shen
EAEslam Abdelrahman
Eslam Abdelrahman
ESEssam Sleiman
Essam Sleiman
MZMingchen Zhuge
Mingchen Zhuge
MEMohamed Elhoseiny
Mohamed Elhoseiny

30 May 2022 in JMIR Biomedical Engineering
Verified
Journal Article
Open access
ESEssam Sleiman
Essam Sleiman
OMOnur Cezmi Mutlu
Onur Cezmi Mutlu
SSSaimourya Surabhi
Saimourya Surabhi
AHArman Husic
Arman Husic
AKAaron Kline
Aaron Kline
DWDennis Wall
Dennis Wall
Journal of Medical Internet Research - International Scientific Journal for Medical Research, Information and Communication on the Internet #Preprint #PeerReviewMe: Warning: This is a unreviewed preprint. Readers are warned that the document has not been peer-reviewed by expert/patient reviewers or an academic editor, may contain misleading claims, and is likely to undergo changes before final publication, if accepted, or may have been rejected/withdrawn. Readers with interest and expertise are encouraged to sign up as peer-reviewer, if the paper is within an open peer-review period. Please cite this preprint only for review purposes or for grant applications and CVs (if you are the author). Background: Autism spectrum disorder (ASD) is a prevalent neurodevelopmental disorder encountered by 1 in 44 children in the United States. Patients with autism face difficulty effectively communicating with peers, articulating feelings and emotions, and controlling behaviors. Rapid and early diagnosis leads to improved treatment outcomes; however, current medical techniques often take years. Difficult, time-consuming diagnoses accompanied by the recent advances in computer vision have led to a surge of interest from researchers in developing models to streamline the autism diagnosis process. Objective: We aimed to assess the viability of at-home autism diagnosis by leveraging video data collected from a mobile game app to train a computer vision model using insights extracted from the change in emotion-related expression features over time. Methods: With our GuessWhat game-based mobile app, we collected a video data set of 74 children with ASD and neurotypical diagnoses actively playing in a natural home environment. To investigate and quantify the significance of facial emotion features for ASD detection, we developed a deep learning–based autism classifier with the following two components: a convolutional neural network (CNN) attached to a long short-term memory (LSTM). We pretrained our CNN backbone in the following two ways: (1) on ImageNet or (2) on a compilation of facial expression recognition data sets to analyze autism detection performance improvements using emotion features. The output of each CNN is fed into an LSTM model to diagnose ASD from video data. Results: Our top-performing architecture employing a CNN backbone trained on the facial expression recognition data sets and a unidirectional LSTM achieved a top accuracy of 91.2%, precision of 91.2%, recall of 89.8%, and an F1-score of 90.6%. Conclusions: We discovered the change in an individual’s facial expression features over time as a relevant marker for autism detection. Extracting emotional features from each frame resulted in a 27.8% F1-score improvement when compared to ImageNet weights. Our study demonstrates the capability of mobile apps to collect a natural, diverse data set for an improved autism diagnosis. These results demonstrate that deep learning and computer vision–based methods are instrumental for automated autism diagnosis from at-home recorded videos using unspecialized equipment.

17 July 2024 in arXiv
Verified
Preprint
Open access
PDF file
KAKirolos Ataallah
Kirolos Ataallah
XSXiaoqian Shen
Xiaoqian Shen
EAEslam Abdelrahman
Eslam Abdelrahman
ESEssam Sleiman
Essam Sleiman
MZMingchen Zhuge
Mingchen Zhuge
MEMohamed Elhoseiny
Mohamed Elhoseiny
Most current LLM-based models for video understanding can process videos within minutes. However, they struggle with lengthy videos due to challenges such as "noise and redundancy", as well as "memory and computation" constraints. In this paper, we present Goldfish, a methodology tailored for comprehending videos of arbitrary lengths. We also introduce the TVQA-long benchmark, specifically designed to evaluate models' capabilities in understanding long videos with questions in both vision and text content. Goldfish approaches these challenges with an efficient retrieval mechanism that initially gathers the top-k video clips relevant to the instruction before proceeding to provide the desired response. This design of the retrieval mechanism enables the Goldfish to efficiently process arbitrarily long video sequences, facilitating its application in contexts such as movies or television series. To facilitate the retrieval process, we developed MiniGPT4-Video that generates detailed descriptions for the video clips. In addressing the scarcity of benchmarks for long video evaluation, we adapted the TVQA short video benchmark for extended content analysis by aggregating questions from entire episodes, thereby shifting the evaluation from partial to full episode comprehension. We attained a 41.78% accuracy rate on the TVQA-long benchmark, surpassing previous methods by 14.94%. Our MiniGPT4-Video also shows exceptional performance in short video comprehension, exceeding existing state-of-the-art methods by 3.23%, 2.03%, 16.5% and 23.59% on the MSVD, MSRVTT, TGIF, and TVQA short video benchmarks, respectively. These results indicate that our models have significant improvements in both long and short-video understanding. Our models and code have been made publicly available at https://vision-cair.github.io/Goldfish_website/

30 May 2022 in JMIR Preprints
Verified
Preprint
Open access
ESEssam Sleiman
Essam Sleiman
OMOnur Cezmi Mutlu
Onur Cezmi Mutlu
SSSaimourya Surabhi
Saimourya Surabhi
AHArman Husic
Arman Husic
AKAaron Kline
Aaron Kline
DWDennis Wall
Dennis Wall
BACKGROUND Autism Spectrum Disorder (ASD) is a prevalent neurodevelopmental disorder encountered by 1 in 44 children in the United States of America. Autism patients face difficulty effectively communicating with peers, articulating feelings and emotions, and controlling behaviors. Rapid and early diagnosis leads to improved treatment outcomes, however current medical techniques often take years. Difficult, time-consuming diagnoses accompanied by the recent advances in computer vision have led to a surge of interest from researchers in developing models to streamline the autism diagnosis process. OBJECTIVE We aim to assess the viability of at-home autism diagnosis by leveraging video data collected from a mobile game app to train a computer vision model utilizing insights extracted from the change in emotion expression features over time. METHODS With our GuessWhat game-based mobile app, we collect a video dataset of 74 ASD and NT children actively playing in a natural home environment. To investigate and quantify the significance of facial emotion features for ASD detection, we develop a deep learning-based autism classifier with two components: a Convolutional Neural Network (CNN) attached to a Long Short Term Memory (LSTM). We pre-train our CNN backbone in two fashions: (1) on ImageNet or (2) on a compilation of Facial Expression Recognition (FER) datasets to analyze autism detection performance improvements utilizing emotion features. The output of each CNN is fed into an LSTM model to diagnose ASD from video data. RESULTS Our top-performing architecture utilizing a CNN backbone pre-trained on ImageNet obtained an accuracy of 45.8% and an F1 score of 62.8% while our corresponding top architecture employing a CNN backbone trained on the FER datasets achieved top accuracy of 91.2% and an F1 score of 90.6%. CONCLUSIONS We discovered the change in an individual's facial expression features over time as a relevant marker for autism detection. Extracting emotional features from each frame resulted in a 27.8% F1 score improvement when compared to non-emotion weights. Our study demonstrates the capability of mobile applications to collect a natural, diverse dataset for an improved autism diagnosis. These results demonstrate Deep Learning and Computer Vision based methods are instrumental for automated autism diagnosis from at-home recorded videos using unspecialized equipment.