Zechen Ma

Dr. Zechen Ma

orcid

Research Focus & Metrics

Academic Career Length

10Years (2017-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
18 January 2024 in Arthritis & Rheumatology
Verified
Journal Article
Open access
CYCarrie Ye
Carrie Ye
EZElric Zweck
Elric Zweck
ZMZechen Ma
Zechen Ma
JSJustin Smith
Justin Smith
SKSteven Katz
Steven Katz
Objective: The objective of the current study was to assess the quality of large language model (LLM) chatbot versus physician‐generated responses to patient‐generated rheumatology questions.Methods: We conducted a single‐center cross‐sectional survey of rheumatology patients (n = 17) in Edmonton, Alberta, Canada. Patients evaluated LLM chatbot versus physician‐generated responses for comprehensiveness and readability, with four rheumatologists also evaluating accuracy by using a Likert scale from 1 to 10 (1 being poor, 10 being excellent).Results: Patients rated no significant difference between artificial intelligence (AI) and physician‐generated responses in comprehensiveness (mean 7.12 ± SD 0.99 vs 7.52 ± 1.16; P = 0.1962) or readability (7.90 ± 0.90 vs 7.80 ± 0.75; P = 0.5905). Rheumatologists rated AI responses significantly poorer than physician responses on comprehensiveness (AI 5.52 ± 2.13 vs physician 8.76 ± 1.07; P < 0.0001), readability (AI 7.85 ± 0.92 vs physician 8.75 ± 0.57; P = 0.0003), and accuracy (AI 6.48 ± 2.07 vs physician 9.08 ± 0.64; P < 0.0001). The proportion of preference to AI‐ versus physician‐generated responses by patients and physicians was 0.45 ± 0.18 and 0.15 ± 0.08, respectively (P = 0.0106). After learning that one answer for each question was AI generated, patients were able to correctly identify AI‐generated answers at a lower proportion compared to physicians (0.49 ± 0.26 vs 0.97 ± 0.04; P = 0.0183). The average word count of AI answers was 69.10 ± 25.35 words, as compared to 98.83 ± 34.58 words for physician‐generated responses (P = 0.0008).Conclusion: Rheumatology patients rated AI‐generated responses to patient questions similarly to physician‐generated responses in terms of comprehensiveness, readability, and overall preference. However, rheumatologists rated AI responses significantly poorer than physician‐generated responses, suggesting that LLM chatbot responses are inferior to physician responses, a difference that patients may not be aware of. image

12 January 2018 in Disability and Rehabilitation
Verified
Journal Article
Closed access
ZMZechen Ma
Zechen Ma
MBMark Theodore Bayley
Mark Theodore Bayley
LPLaure Perrier
Laure Perrier
PDPriya Dhir
Priya Dhir
LDLana Dépatie
Lana Dépatie
SMSarah Munce
Sarah Munce
Background: Adverse childhood experiences are significant risk factors for physical and mental illnesses in adulthood. Traumatic brain injury/concussion is a challenging condition where pre-injury factors may affect recovery. The association between childhood adversity and traumatic brain injury/concussion has not been previously reviewed. The research question addressed is: What is known from the existing literature about the association between adverse childhood experiences and traumatic brain injury/concussion in adults? Methods: All original studies of any type published in English since 2007 on adverse childhood experiences and traumatic brain injury/concussion outcomes were included. The literature search was conducted in multiple electronic databases. Arksey and O’Malley and Levac et al.’s scoping review frameworks were used. Two reviewers independently completed screening and data abstraction. Results: The review yielded six observational studies. Included studies were limited to incarcerated or homeless samples, and individuals at high-risk of or with mental illnesses. Across studies, methods for childhood adversity and traumatic brain injury/concussion assessment were heterogeneous. Discussion: A positive association between adverse childhood experiences and traumatic brain injury occurrence was identified. The review highlights the importance of screening and treatment of adverse childhood experiences. Future research should extend to the general population and implications on injury recovery.

28 June 2023 in Diagnostics
Verified
Journal Article
Open access
PDF file
ZMZechen Ma
Zechen Ma
DMDouwe Johannes Mulder
Douwe Johannes Mulder
Robert GniadeckiRobert Gniadecki
Robert Gniadecki
Jan Willem Cohen TervaertJan Willem Cohen Tervaert
Jan Willem Cohen Tervaert
MOMohammed S. Osman
Mohammed S. Osman
Introduction: Nailfolds of patients with systemic sclerosis (SSc) provide an opportunity to directly visualize microvascular remodeling in SSc. Nailfold video capillaroscopy (NVC) remains the gold standard for assessing nailfold capillaroscopy (NFC). However, access to NVC is limited by expense and expertise. This review aims to synthesize current research on other NFC devices compared to NVC. Methods: The literature search included the primary research of adult patients with SSc as defined by the 2013 ACR/EULAR criteria. Methods of assessing NFC included stereomicroscopy/wide-field microscopy, ophthalmoscopy, dermatoscopy, smartphone devices, and digital USB microscopy. Primary outcomes included both qualitative (normal vs. abnormal nailfolds, overall pattern recognition, presence/absence of giant capillaries, hemorrhages, and abnormal morphology) and quantitative (capillary density and dimension) measures. Results: The search yielded 471 studies, of which 9 were included. Five studies compared NVC to dermatoscopy, two compared it to widefield/stereomicroscopy, one to smartphone attachments, and one to USB microscopy. In dermatoscopy studies, NVC had a higher percentage of images that were interpretable (63–77% vs. 100%), classifiable (70% vs. 84%), or gradable (70% vs. 79.3%) across three studies. Dermatoscopy had a lower sensitivity (60.2% vs. 81.6%) and higher specificity (92.5% vs. 84.6%) compared to NVC. One stereomicroscopy study found a significant difference between methods in capillary density in limited cutaneous SSc, while another found correlations in all parameters between stereomicroscopy and NVC. One smartphone lens had good agreement with NVC on abnormal capillary morphology and density. USB microscopy was able to differentiate between SSc and healthy controls using mean capillary width but not by capillary density. Discussion: A dermatoscope may serve as a more portable and affordable screening tool to identify a normal “scleroderma pattern”, and images that need further corroboration by NVC. NFC parameters reported are heterogenous and the standardization of these parameters is important, especially in non-gold-standard devices.

11 October 2017 in BMJ Open
Verified
Journal Article
Open access
PDF file
ZMZechen Ma
Zechen Ma
MBMark Theodore Bayley
Mark Theodore Bayley
LPLaure Perrier
Laure Perrier
PDPriya Dhir
Priya Dhir
LDLana Dépatie
Lana Dépatie
SMSarah Munce
Sarah Munce
Introduction: Exposure to adverse childhood experiences (ACEs) is a significant risk factor for physical and mental illnesses later in life. Concussion or traumatic brain injury is a challenging condition where preinjury factors may interact to affect recovery. The association between ACEs and traumatic brain injury/concussion is not well mapped in any previous reviews of the literature. Using a scoping review methodology, the research question that will be addressed is: what is known from the existing literature about the association between ACEs and traumatic brain injury/concussion in adults? Methods and analysis: The methodological frameworks outlined by Arksey and O’Malley and Levac et al will be used. All original studies in English published since 2007 investigating ACEs and traumatic brain injury/concussion outcomes will be included with no limitations on study type. Literature search strategies will be developed using medical subject headings and text words related to ACEs and traumatic brain injury/concussions. Multiple electronic databases will be searched. Two independent reviewers will screen titles and abstracts for full-text review and full texts for final inclusion. Two independent reviewers will extract data on study characteristics for ACE exposure and traumatic brain injury/concussion outcomes. Extracted data will be summarised quantitatively using numerical counts and qualitatively using thematic analysis. Dissemination: This review will identify knowledge gaps on the associations between ACEs and traumatic brain injury/concussion and promote further research. Knowledge translation will occur throughout the review process with dissemination of project findings to stakeholders at the local, national and international levels.

9 December 2021 in BMJ Open Quality
Verified
Journal Article
Open access
PDF file
ZMZechen Ma
Zechen Ma
MPMercedes Camargo Penuela
Mercedes Camargo Penuela
MLMadelyn Law
Madelyn Law
DJDivya Joshi
Divya Joshi
HCHan-Oh Chung
Han-Oh Chung
JTJennifer Ly Tsang
Jennifer Ly Tsang
Background Clinical guidelines suggest that routine assessment, treatment, and prevention of pain, agitation, and delirium (PAD) is essential to improving patient outcomes as delirium is associated with increased mortality and morbidity. Despite the well-established improvements on patient outcomes, adherence to PAD guidelines is poor in community intensive care units (ICU). This quality improvement (QI) project aims to evaluate the impact of a multifaceted and multidisciplinary intervention on PAD management in a Canadian community ICU and to describe the experience of a Canadian community hospital in conducting a QI project. Methods A ten-member PAD advisory committee was formed to develop and implement the intervention. The intervention consisted of a multidisciplinary rounds script, poster, interviews, visual reminders, educational modules, pamphlet and video. The 4-week intervention targeted nurses, family members, physicians, and the multidisciplinary team. An uncontrolled, before-and-after study methodology was used. Adherence to PAD assessment guidelines by nurses was measured over a 6-week pre-intervention and over a 6-week post-intervention periods. Results Data on 430 and 406 patient-days (PD) were available for analysis during the pre- and post- intervention periods, respectively. The intervention did not improve the proportion of PD with guideline compliance to the assessment of pain (23.4% vs. 22.4%, p=0.80), agitation (42.9% vs. 38.9%, p=0.28), nor delirium (35.2% vs. 29.6%, p=0.10) by nurses. Discussion The implementation of a multifaceted and multidisciplinary intervention on PAD assessment did not result in significant improvements in guideline adherence in a community ICU. Barriers to knowledge translation are apparent at multiple levels including the personal level (low completion rates on educational modules), interventional level (under-collection of data), and organisational level (coinciding with hospital accreditation education). Our next steps include reintroduction of education modules using organisation approved platforms, updating existing ICU policy, updating admission order sets, and conducting audit and feedback.

8 April 2026 in Journal of Medical Internet Research
Verified
Journal Article
Open access
CYCarrie Ye
Carrie Ye
JMJoseph Ross Mitchell
Joseph Ross Mitchell
DBDaniel C Baumgart
Daniel C Baumgart
ZMZechen Ma
Zechen Ma
AFAngela Lim Fung
Angela Lim Fung
MAMohamed Abdalla
Mohamed Abdalla
Background: Despite the transformative potential of large language models (LLMs) in health care, the rapid development of these tools has outpaced their rigorous evaluation. While artificial intelligence–specific reporting guidelines have been developed to address standardized reporting of artificial intelligence studies, there is currently no specific tool available for risk of bias assessment of LLM question-answer (QA) studies. Existing risk-of-bias tools for medical research are not well suited to the unique challenges of evaluating LLM-QA studies, which creates a critical gap in assessing their safety and effectiveness. Objective: This study aims to develop the Alberta Quality Assessment Tool: Risk of Bias (AQAT:RoB) for LLM-QA studies to systematically evaluate the validity and risk of bias in LLM-QA studies. Methods: We conducted 2 literature reviews. The first was on quality assessment tools for LLM-QA studies, and the second was on LLM-QA studies, which informed the first draft of the AQAT:RoB. The draft AQAT:ROB was further refined through a prespecified iterative process of modified Delphi, consensus meeting, and validation. The first Delphi process occurred between May 1 and May 20, 2025, and the first consensus meeting was held on May 22. The first round of validation was completed by 4 evaluators, who were not part of the consensus meeting, on 16 randomly selected studies. As this first round of validation surpassed our a priori threshold of ≥80% agreement and a Cohen κ of ≥0.61 between evaluators, no further rounds of development and validation were undertaken. A second Delphi process occurred between February 20 and February 23, 2026, to vote on postpilot changes in response to peer review. Results: The AQAT:RoB consists of 5 high-level domains (Questions, Reference Answers, LLM Answers, Evaluators, Outcomes). These domains are subdivided into 9 subdomains. Each subdomain includes at least one “Support for Judgment” and at least one “Type of Bias” and is to be rated “low,” “high,” or “unclear” for risk of bias. A pilot evaluation was completed by internal validators who were not part of the consensus discussion and were asked to complete the AQAT:RoB form for each assigned study. Each of the 16 studies was evaluated by 2 evaluators independently. Pilot validation showed a percent agreement of 86.1% and a Cohen κ of 0.70 between assessors. Conclusions: The AQAT:RoB demonstrates promising initial reliability for assessing the validity or risk of bias in LLM-QA studies. The tool will benefit from future refinements, external validation, and periodic updates to keep pace with evolving technology.