Jeffrey Michael Franc

Jeffrey Michael Franc

Auto-generated profile, based on publicly available information

Jeffrey Michael Franc

University of Alberta

Research Focus & Metrics

Academic Career Length

15Years (2012-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
17 June 2023 in Methodological Innovations
Verified
Journal Article
Open access
JFJeffrey Michael Franc
Jeffrey Michael Franc
KHKevin Kei Ching Hung
Kevin Kei Ching Hung
APAlessandro Pirisi
Alessandro Pirisi
EWEric S Weinstein
Eric S Weinstein
The Delphi technique is a unique survey method that involves an iterative process to gain consensus when consensus is challenging to establish. Survey participants typically rate a variety of statements using a specified rating scale. The survey is repeated for several rounds, and at each round statements that do not reach a predefined level of consensus are advanced to the next round while giving the participants information about the responses of other participants for their comparison. The final statements are then ranked in order of the average rating. The statistical methods to analyze Delphi studies are not well described. This study investigates the use of a 1–7 linear rating scale along with parametric summary statistics for assessment of consensus and ranking of statements. A study set of 9297 individual ratings on the 1–7 scale were obtained from previously performed Delphi studies and used to create 490,000 simulated Delphi ratings with various numbers of participants. While the overall distribution of ratings was strongly left skewed the sampling distribution was near normally distributed for studies with five or more participants. The average difference between the standard deviation and interquartile range was −0.26/7. The overall risk of falsely concluding consensus using the standard deviation as a summary statistic was 7.3% when compared to using the interquartile range. The average difference between mean and median was −0.20/7. The risk of falsely ranking the statements by a value of 0.5 or more was near zero for all sample sizes when the mean was compared to the median. This study suggests that the use of the 1–7 linear rating scale in combination with the parametric summary statistics of standard deviation and mean is a valid method to analyze ratings from Delphi studies.

11 January 2024 in Canadian Journal of Emergency Medicine
Verified
Journal Article
Closed access
PDF file
JFJeffrey Michael Franc
Jeffrey Michael Franc
LCLenard Cheng
Lenard Cheng
AHAlexander Hart
Alexander Hart
RHRyan Hata
Ryan Hata
AHAtilla Hertelendy
Atilla Hertelendy

30 September 2024 in Journal of Medical Internet Research
Verified
Journal Article
Open access
JFJeffrey Micheal Franc
Jeffrey Micheal Franc
Attila HertelendyAttila Hertelendy
Attila Hertelendy
LCLenard Cheng
Lenard Cheng
RHRyan Hata
Ryan Hata
MVManuela Verde
Manuela Verde
Background: The release of ChatGPT (OpenAI) in November 2022 drastically reduced the barrier to using artificial intelligence by allowing a simple web-based text interface to a large language model (LLM). One use case where ChatGPT could be useful is in triaging patients at the site of a disaster using the Simple Triage and Rapid Treatment (START) protocol. However, LLMs experience several common errors including hallucinations (also called confabulations) and prompt dependency. Objective: This study addresses the research problem: “Can ChatGPT adequately triage simulated disaster patients using the START protocol?” by measuring three outcomes: repeatability, reproducibility, and accuracy. Methods: Nine prompts were developed by 5 disaster medicine physicians. A Python script queried ChatGPT Version 4 for each prompt combined with 391 validated simulated patient vignettes. Ten repetitions of each combination were performed for a total of 35,190 simulated triages. A reference standard START triage code for each simulated case was assigned by 2 disaster medicine specialists (JMF and MV), with a third specialist (LC) added if the first two did not agree. Results were evaluated using a gage repeatability and reproducibility study (gage R and R). Repeatability was defined as variation due to repeated use of the same prompt. Reproducibility was defined as variation due to the use of different prompts on the same patient vignette. Accuracy was defined as agreement with the reference standard. Results: Although 35,102 (99.7%) queries returned a valid START score, there was considerable variability. Repeatability (use of the same prompt repeatedly) was 14% of the overall variation. Reproducibility (use of different prompts) was 4.1% of the overall variation. The accuracy of ChatGPT for START was 63.9% with a 32.9% overtriage rate and a 3.1% undertriage rate. Accuracy varied by prompt with a maximum of 71.8% and a minimum of 46.7%. Conclusions: This study indicates that ChatGPT version 4 is insufficient to triage simulated disaster patients via the START protocol. It demonstrated suboptimal repeatability and reproducibility. The overall accuracy of triage was only 63.9%. Health care professionals are advised to exercise caution while using commercial LLMs for vital medical determinations, given that these tools may commonly produce inaccurate data, colloquially referred to as hallucinations or confabulations. Artificial intelligence–guided tools should undergo rigorous statistical evaluation—using methods such as gage R and R—before implementation into clinical settings.

17 December 2021 in Prehospital and Disaster Medicine
Verified
Journal Article
Closed access
PDF file
JFJeffrey M. Franc
Jeffrey M. Franc
SKScott W. Kirkland
Scott W. Kirkland
UWUirá D. Wisnesky
Uirá D. Wisnesky
SCSandra Campbell
Sandra Campbell
BRBrian H. Rowe
Brian H. Rowe
Introduction: The goal of disaster triage at both the prehospital and in-hospital level is to maximize resources and optimize patient outcomes. Of the disaster-specific triage methods developed to guide health care providers, the Simple Triage and Rapid Treatment (START) algorithm has become the most popular system world-wide. Despite its appeal and global application, the accuracy and effectiveness of the START protocol is not well-known.Objectives: The purpose of this meta-analysis was two-fold: (1) to estimate overall accuracy, under-triage, and over-triage of the START method when used by providers across a variety of backgrounds; and (2) to obtain specific accuracy for each of the four START categories: red, yellow, green, and black.Methods: A systematic review and meta-analysis was conducted that searched Medline (OVID), Embase (OVID), Global Health (OVID), CINAHL (EBSCO), Compendex (Engineering Village), SCOPUS, ProQuest Dissertations and Theses Global, Cochrane Library, and PROSPERO. The results were expanded by hand searching of journals, reference lists, and the grey literature. The search was executed in March 2020. The review considered the participants, interventions, context, and outcome (PICO) framework and followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Accuracy outcomes are presented as means with 95% confidence intervals (CI) as calculated using the binomial method. Pooled meta-analyses of accuracy outcomes using fixed and random effects models were calculated and the heterogeneity was assessed using the Q statistic.Results: Thirty-two studies were included in the review, most of which utilized a non-randomized study design (84%). Proportion of victims correctly triaged using START ranged from 0.27 to 0.99 with an overall triage accuracy of 0.73 (95% CI, 0.67 to 0.78). Proportion of over-triage was 0.14 (95% CI, 0.11 to 0.17) while the proportion of under-triage was 0.10 (95% CI, 0.072 to 0.14). There was significant heterogeneity of the studies for all outcomes (P < .0001).Conclusion: This meta-analysis suggests that START is not accurate enough to serve as a reliable disaster triage tool. Although the accuracy of START may be similar to other models of disaster triage, development of a more accurate triage method should be urgently pursued.

9 February 2016 in Prehospital and Disaster Medicine
Verified
Journal Article
Closed access
PDF file
CLChristopher W.C. Lee
Christopher W.C. Lee
SMShelley L McLeod
Shelley L McLeod
KAKristine Van Aarsen
Kristine Van Aarsen
MKMichelle Klingel
Michelle Klingel
JFJeffrey M. Franc
Jeffrey M. Franc
MPMichael B. Peddle
Michael B. Peddle
Introduction: During mass-casualty incidents (MCIs), patient volume often overwhelms available Emergency Medical Services (EMS) personnel. First responders are expected to triage, treat, and transport patients in a timely fashion. If other responders could triage accurately, prehospital EMS resources could be focused more directly on patients that require immediate medical attention and transport.Hypothesis: Triage accuracy, error patterns, and time to triage completion are similar between second-year primary care paramedic (PCP) and fire science (FS) students participating in a simulated MCI using the Sort, Assess, Life-saving interventions, Treatment/Transport (SALT) triage algorithm.Methods: All students in the second-year PCP program and FS program at two separate community colleges were invited to participate in this study. Immediately following a 30-minute didactic session on SALT, participants were given a standardized briefing and asked to triage an eight-victim, mock MCI using SALT. The scenario consisted of a four-car motor vehicle collision with each victim portrayed by volunteer actors given appropriate moulage and symptom coaching for their pattern of injury. The total number and acuity of victims were unknown to participants prior to arrival to the mock scenario.Results: Thirty-eight PCP and 29 FS students completed the simulation. Overall triage accuracy was 79.9% for PCP and 72.0% for FS (∆ 7.9%; 95% CI, 1.2-14.7) students. No significant difference was found between the groups regarding types of triage errors. Over-triage, under-triage, and critical errors occurred in 10.2%, 7.6%, and 2.3% of PCP triage assignments, respectively. Fire science students had a similar pattern with 15.2% over-triaged, 8.7% under-triaged, and 4.3% critical errors. The median [IQR] time to triage completion for PCPs and FSs were 142.1 [52.6] seconds and 159.0 [40.5] seconds, respectively (P=.19; Mann-Whitney Test).Conclusions: Primary care paramedics performed MCI triage more accurately than FS students after brief SALT training, but no difference was found regarding types of error or time to triage completion. The clinical importance of this difference in triage accuracy likely is minimal, suggesting that fire services personnel could be considered for MCI triage depending on the availability of prehospital medical resources and appropriate training. Lee CWC , McLeod SL , Van Aarsen K , Klingel M , Franc JM , Peddle MB . First responder accuracy using SALT during mass-casualty incident simulation. Prehosp Disaster Med. 2016;31(2):150–154.

1 February 2012 in Prehospital and Disaster Medicine
Verified
Journal Article
Closed access
PDF file
LALeslie Ahlborn
Leslie Ahlborn
JFJeffrey Michael Franc
Jeffrey Michael Franc
DMD Sport Med
D Sport Med
Background: The state of Oklahoma, known for destructive tornados, has a native Spanish-speaking (NSS) population of approximately 180,241, of which 50% report being able to speak English “very well” (US Census Bureau). With almost 50% of these native Spanish-speaking persons being limited English proficient (LEP), their reception of tornado hazard communications may be restricted. This study conducted in northeast Oklahoma (USA) evaluates the association between native language and receiving tornado hazard communications.Methods: This study was a cross-sectional survey conducted among a convenience sample of NSS and native English-speaking (NES) adults at Xavier Clinic and St. Francis Trauma Emergency Center in Tulsa, OK, USA from September 2009 through December 2009. Of the 82 surveys administered, 80 were returned, with 40 NES and 40 NSS participants. A scoring system (Severe Weather Information Reception (SWIR)) was developed to quantify reception of hazard information among the study participants (1–3 points = poor reception, 4–5 = adequate reception, 6–8 = excellent reception). Pearson’s chi-squared test was used to calculate differences between groups with Yates’ continuity correction applied where appropriate, and SWIR scores were analyzed using ANOVA. P-values <.05 were considered significant.Results: NSS fluency in English was 25.6%. No significant association was found between native language and those who watch television, listen to radio, have a National Oceanic and Atmospheric Administration (NOAA) All Hazards radio or telephone, or are in audible range of a tornado siren. NSS were less likely to have Internet access (P < .004), and less likely to know of local telephone warning programs (P < .03). The mean NSS SWIR score was 3.2 (95% CI, 2.8-3.7) while LEP NSS averaged 2.8 (95% CI, 2.4-3.2). The mean NES SWIR score was 4.5 (95% CI, 4.1-5.0).Conclusion: Results demonstrate a disparity in tornado warning reception between NSS and NES. Poor English proficiency was noted to be 75% among NSS, which is approximately 25% more than estimated by the US Census Bureau. This study demonstrates a need for emergency managers to recognize when appropriate and overcome communication disparities among limited English proficient populations.