Hesam Mahmoudi

Hesam Mahmoudi

Auto-generated profile, based on publicly available information

Hesam Mahmoudi

Harvard University - Harvard Medical School

Research Focus & Metrics

Academic Career Length

7Years (2020-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
21 March 2023 in Journal of Science Education and Technology
Verified
Journal Article
Open access
PDF file
KDKirsten A. Davis
Kirsten A. Davis
DGDustin Grote
Dustin Grote
HMHesam Mahmoudi
Hesam Mahmoudi
LPLogan Perry
Logan Perry
NGNavid Ghaffarzadegan
Navid Ghaffarzadegan
KTKonstantinos Triantis
Konstantinos Triantis
Self-report assessments are used frequently in higher education to assess a variety of constructs, including attitudes, opinions, knowledge, and competence. Systems thinking is an example of one competence often measured using self-report assessments where individuals answer several questions about their perceptions of their own skills, habits, or daily decisions. In this study, we define systems thinking as the ability to see the world as a complex interconnected system where different parts can influence each other, and the interrelationships determine system outcomes. An alternative, less-common, assessment approach is to measure skills directly by providing a scenario about an unstructured problem and evaluating respondents’ judgment or analysis of the scenario (scenario-based assessment). This study explored the relationships between engineering students’ performance on self-report assessments and scenario-based assessments of systems thinking, finding that there were no significant relationships between the two assessment techniques. These results suggest that there may be limitations to using self-report assessments as a method to assess systems thinking and other competencies in educational research and evaluation, which could be addressed by incorporating alternative formats for assessing competence. Future work should explore these findings further and support the development of alternative assessment approaches.

1 April 2020 in System Dynamics Review
Verified
Journal Article
Closed access
PDF file
KDKirsten Davis
Kirsten Davis
NGNavid Ghaffarzadegan
Navid Ghaffarzadegan
JGJacob Grohs
Jacob Grohs
DGDustin Grote
Dustin Grote
NHNiyousha Hosseinichimeh
Niyousha Hosseinichimeh
KTKonstantinos Triantis
Konstantinos Triantis

11 September 2025 in JMIR AI
Verified
Journal Article
Open access
HMHesam Mahmoudi
Hesam Mahmoudi
DCDoris Chang
Doris Chang
HLHannah Lee
Hannah Lee
NGNavid Ghaffarzadegan
Navid Ghaffarzadegan
MJMohammad S. Jalali
Mohammad S. Jalali
Background: Systematic literature reviews (SLRs) are foundational for synthesizing evidence across diverse fields and are especially important in guiding research and practice in health and biomedical sciences. However, they are labor intensive due to manual data extraction from multiple studies. As large language models (LLMs) gain attention for their potential to automate research tasks and extract basic information, understanding their ability to accurately extract explicit data from academic papers is critical for advancing SLRs. Objective: Our study aimed to explore the capability of LLMs to extract both explicitly outlined study characteristics and deeper, more contextual information requiring nuanced evaluations, using ChatGPT (GPT-4). Methods: We screened the full text of a sample of COVID-19 modeling studies and analyzed three basic measures of study settings (ie, analysis location, modeling approach, and analyzed interventions) and three complex measures of behavioral components in models (ie, mobility, risk perception, and compliance). To extract data on these measures, two researchers independently extracted 60 data elements using manual coding and compared them with the responses from ChatGPT to 420 queries spanning 7 iterations. Results: ChatGPT’s accuracy improved as prompts were refined, showing improvements of 33% and 23% between the initial and final iterations for extracting study settings and behavioral components, respectively. In the initial prompts, 26 (43.3%) of 60 ChatGPT responses were correct. However, in the final iteration, ChatGPT extracted 43 (71.7%) of the 60 data elements, showing better performance in extracting explicitly stated study settings (28/30, 93.3%) than in extracting subjective behavioral components (15/30, 50%). Nonetheless, the varying accuracy across measures highlighted its limitations. Conclusions: Our findings underscore LLMs’ utility in extracting basic as well as explicit data in SLRs by using effective prompts. However, the results reveal significant limitations in handling nuanced, subjective criteria, emphasizing the necessity for human oversight.

1 January 2024 in SSRN Electronic Journal
Verified
Preprint
Open access
PDF file
HMHesam Mahmoudi
Hesam Mahmoudi
DCDoris Chang
Doris Chang
HLHannah Lee
Hannah Lee
NGNavid Ghaffarzadegan
Navid Ghaffarzadegan
MJMohammad S. Jalali
Mohammad S. Jalali
Objectives: To assess the effectiveness of large language models (LLMs) for systematic reviews, focusing on measures from basic details to complex criteria requiring nuanced evaluations. Methods: Screening the full text of 10 COVID-19 modeling studies, we analyzed three basic measures of study settings, e.g., analysis location, and three complex measures of behavioral components in models, e.g., risk perception. To extract data on these measures, we conducted 60 manual codings and compared them with 420 queries spanning seven iterations in ChatGPT. Results: ChatGPT demonstrated 72% overall accuracy in extracting 60 data elements, performing better in extracting explicitly stated study settings (93%) than subjective behavioral components (50%). Discussion: While ChatGPT’s accuracy improved as prompts were refined, varying accuracy across measures highlights its limitations. Conclusion: We underscore LLMs’ utility in systematic reviews for basic, explicit data extraction but reveal significant limitations in handling nuanced, subjective criteria, emphasizing the current necessity for human oversight.

27 June 2024 in Systems Research and Behavioral Science
Verified
Journal Article
Open access
NLNing‐Yuan Georgia Liu
Ning‐Yuan Georgia Liu
HMHesam Mahmoudi
Hesam Mahmoudi
KTKonstantinos Triantis
Konstantinos Triantis
NGNavid Ghaffarzadegan
Navid Ghaffarzadegan
Systems thinking (ST) includes a set of critical skills and approaches for addressing today's complex societal problems. Therefore, it has been introduced into the curricula of many educational programmes around the world. Despite all the attention to ST, there is less consensus when it comes to evaluating and assessing ST skills. Particularly, a quantitative assessment approach that captures ST's multi‐dimensionality is crucial when evaluating the degree to which one has learned and utilizes ST. This paper proposes a systematic approach to create such a multi‐dimensional Index of ST from textual data. Initially, we provide an overview of the theoretical background as it pertains to different measurement approaches of ST skills. Then we provide a conceptual framework based on ST skill measures and transform this framework into a quantifiable model. Finally, using student data, we provide an illustration of an integrated index of ST skills. We compute this index by using a mixed methods approach, including robust principal component analysis, data envelopment analysis and two‐staged bootstrapping approach. The results show that (i) our model serves as a systematic multi‐dimensional ST approach by including multiple measures of ST skills and (ii) international students and self‐reported math skills are found as significant predictors of one's level of ST in the graduate student dataset ( N = 30), however no significant factors are found in the first‐year engineering student dataset ( N = 144).

1 January 2024 in SSRN Electronic Journal
Verified
Preprint
Open access
PDF file
NLNingYuan Liu
NingYuan Liu
HMHesam Mahmoudi
Hesam Mahmoudi
KTKonstantinos Triantis
Konstantinos Triantis
NGNavid Ghaffarzadegan
Navid Ghaffarzadegan
Systems Thinking (ST) includes a set of critical skills and approaches for addressing today’s complex societal problems. Therefore, it has been introduced into the curricula of many educational programs around the world. Despite all the attention to ST, there is less consensus when it comes to evaluating and assessing ST skills. Particularly, a quantitative assessment approach that captures ST’s multidimensionality is crucial when evaluating the degree to which one has learned and utilizes ST. This paper proposes a systematic approach to create such a Multi-dimensional Index of ST from textual data. Initially, we provide an overview of the theoretical background as it pertains to different measurement approaches of ST skills. Then we provide a conceptual framework based on ST skill measures and transform this framework into a quantifiable model. Finally, using student data, we provide an illustration of an integrated index of ST skills. We compute this index by using a mixed methods approach, including Robust Principal Component Analysis, Data Envelopment Analysis, and two-staged bootstrapping approach. The results show that: (i) Our model serves as a systematic multi-dimensional ST approach by including multiple measures of ST skills, and (ii) international students and self-reported math skills are found as significant predictors of one’s level of ST in the graduate student dataset (N=30), however no significant factors are found in the first-year engineering student dataset (N =144).