Martin Holderried

Martin Holderried

Auto-generated profile, based on publicly available information

Martin Holderried

Department of Otolaryngology—Head and Neck Surgery, University of Tübingen Medical Center, Tübingen, Germany

Research Focus & Metrics

Academic Career Length

13Years (2014-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
16 January 2024 in JMIR Medical Education
Verified
Journal Article
Open access
FHFriederike Holderried
Friederike Holderried
CSChristian Stegemann–Philipps
Christian Stegemann–Philipps
LHLea Herschbach
Lea Herschbach
JMJulia-Astrid Moldt
Julia-Astrid Moldt
ANAndrew Nevins
Andrew Nevins
MMMoritz Mahling
Moritz Mahling
Background: Communication is a core competency of medical professionals and of utmost importance for patient safety. Although medical curricula emphasize communication training, traditional formats, such as real or simulated patient interactions, can present psychological stress and are limited in repetition. The recent emergence of large language models (LLMs), such as generative pretrained transformer (GPT), offers an opportunity to overcome these restrictions Objective: The aim of this study was to explore the feasibility of a GPT-driven chatbot to practice history taking, one of the core competencies of communication. Methods: We developed an interactive chatbot interface using GPT-3.5 and a specific prompt including a chatbot-optimized illness script and a behavioral component. Following a mixed methods approach, we invited medical students to voluntarily practice history taking. To determine whether GPT provides suitable answers as a simulated patient, the conversations were recorded and analyzed using quantitative and qualitative approaches. We analyzed the extent to which the questions and answers aligned with the provided script, as well as the medical plausibility of the answers. Finally, the students filled out the Chatbot Usability Questionnaire (CUQ). Results: A total of 28 students practiced with our chatbot (mean age 23.4, SD 2.9 years). We recorded a total of 826 question-answer pairs (QAPs), with a median of 27.5 QAPs per conversation and 94.7% (n=782) pertaining to history taking. When questions were explicitly covered by the script (n=502, 60.3%), the GPT-provided answers were mostly based on explicit script information (n=471, 94.4%). For questions not covered by the script (n=195, 23.4%), the GPT answers used 56.4% (n=110) fictitious information. Regarding plausibility, 842 (97.9%) of 860 QAPs were rated as plausible. Of the 14 (2.1%) implausible answers, GPT provided answers rated as socially desirable, leaving role identity, ignoring script information, illogical reasoning, and calculation error. Despite these results, the CUQ revealed an overall positive user experience (77/100 points). Conclusions: Our data showed that LLMs, such as GPT, can provide a simulated patient experience and yield a good user experience and a majority of plausible answers. Our analysis revealed that GPT-provided answers use either explicit script information or are based on available information, which can be understood as abductive reasoning. Although rare, the GPT-based chatbot provides implausible information in some instances, with the major tendency being socially desirable instead of medically plausible information.

16 August 2024 in JMIR Medical Education
Verified
Journal Article
Open access
FHFriederike Holderried
Friederike Holderried
CSChristian Stegemann-Philipps
Christian Stegemann-Philipps
AHAnne Herrmann-Werner
Anne Herrmann-Werner
TLTeresa Loda
Teresa Loda
MHMartin Holderried
Martin Holderried
MMMoritz Mahling
Moritz Mahling
Background: Although history taking is fundamental for diagnosing medical conditions, teaching and providing feedback on the skill can be challenging due to resource constraints. Virtual simulated patients and web-based chatbots have thus emerged as educational tools, with recent advancements in artificial intelligence (AI) such as large language models (LLMs) enhancing their realism and potential to provide feedback. Objective: In our study, we aimed to evaluate the effectiveness of a Generative Pretrained Transformer (GPT) 4 model to provide structured feedback on medical students’ performance in history taking with a simulated patient. Methods: We conducted a prospective study involving medical students performing history taking with a GPT-powered chatbot. To that end, we designed a chatbot to simulate patients’ responses and provide immediate feedback on the comprehensiveness of the students’ history taking. Students’ interactions with the chatbot were analyzed, and feedback from the chatbot was compared with feedback from a human rater. We measured interrater reliability and performed a descriptive analysis to assess the quality of feedback. Results: Most of the study’s participants were in their third year of medical school. A total of 1894 question-answer pairs from 106 conversations were included in our analysis. GPT-4’s role-play and responses were medically plausible in more than 99% of cases. Interrater reliability between GPT-4 and the human rater showed “almost perfect” agreement (Cohen κ=0.832). Less agreement (κ<0.6) detected for 8 out of 45 feedback categories highlighted topics about which the model’s assessments were overly specific or diverged from human judgement. Conclusions: The GPT model was effective in providing structured feedback on history-taking dialogs provided by medical students. Although we unraveled some limitations regarding the specificity of feedback for certain feedback categories, the overall high agreement with human raters suggests that LLMs can be a valuable tool for medical education. Our findings, thus, advocate the careful integration of AI-driven feedback mechanisms in medical training and highlight important aspects when LLMs are used in that context.

13 April 2022 in JMIR mHealth and uHealth
Verified
Journal Article
Open access
MHMarietta Hamberger
Marietta Hamberger
NINensi Ikonomi
Nensi Ikonomi
JSJulian Schwab
Julian Schwab
SKSilke Kühlwein
Silke Kühlwein
AFAxel Fürstberger
Axel Fürstberger
HKHans Kestler
Hans Kestler
In its most trending interpretation, empowerment in health care is implemented as a patient-centered approach. In the same sense, many mobile health (mHealth) apps are being developed with a primary focus on the individual user. The integration of mHealth apps into the health care system has the potential to counteract existing challenges, including incomplete or nonstandardized medical data and lack of communication, especially in the intersectional context (eg, patients, medical forces). However, concerns about data security and privacy, regional differences in regulations, lack of accessibility, and nontransparent apps hinder the successful integration of mHealth into the health care system. One approach to address this is to rethink the interpretation of empowerment. On that basis, we here examine existing approaches of individual empowerment and subsequently analyze a different view of empowerment in digital health, namely interaction empowerment. Such a change of perspective could positively influence intersectoral communication and facilitate secure data and knowledge sharing. We discuss this novel viewpoint on empowerment, focusing on more efficient integration and development of mHealth approaches. A renewed interpretation of empowerment could thus buffer current limitations of individual empowerment while also advancing digitization of the health system.

21 January 2025 in Journal of Medical Internet Research
Verified
Journal Article
Open access
FEFelix Eisinger
Felix Eisinger
FHFriederike Holderried
Friederike Holderried
MMMoritz Mahling
Moritz Mahling
CSChristian Stegemann–Philipps
Christian Stegemann–Philipps
AHAnne Herrmann-Werner
Anne Herrmann-Werner
MHMartin Holderried
Martin Holderried
Background: For hospitalized patients, the discharge letter serves as a crucial source of medical information, outlining important discharge instructions and health management tasks. However, these letters are often written in professional jargon, making them difficult for patients with limited medical knowledge to understand. Large language models, such as GPT, have the potential to transform these discharge summaries into patient-friendly letters, improving accessibility and understanding. Objective: This study aims to use GPT-4 to convert discharge letters into more readable patient-centered letters. We evaluated how effectively and comprehensively GPT-4 identified and transferred patient safety–relevant information from the discharge letters to the transformed patient letters. Methods: Three discharge letters were created based on common medical conditions, containing 72 patient safety–relevant pieces of information, referred to as “learning objectives.” GPT-4 was prompted to transform these discharge letters into patient-centered letters. The resulting patient letters were analyzed for medical accuracy, patient centricity, and the ability to identify and translate the learning objectives. Bloom’s taxonomy was applied to analyze and categorize the learning objectives. Results: GPT-4 addressed the majority (56/72, 78%) of the learning objectives from the discharge letters. However, 11 of the 72 (15%) learning objectives were not included in the majority of the patient-centered letters. A qualitative analysis based on Bloom’s taxonomy revealed that learning objectives in the “Understand” category (9/11) were more frequently omitted than those in the “Remember” category (2/11). Most of the missing learning objectives were related to the content field of “prevention of complications.” By contrast, learning objectives regarding “lifestyle” and “organizational” aspects were addressed more frequently. Medical errors were found in a small proportion of sentences (31/787, 3.9%). In terms of patient centricity, the patient-centered letters demonstrated better readability than the discharge letters. Compared with discharge letters, they included fewer medical terms (132/860, 15.3%, vs 165/273, 60/4%), fewer abbreviations (43/860, 5%, vs 49/273, 17.9%), and more explanations of medical terms (121/131, 92.4%, vs 0/165, 0%). Conclusions: Our study demonstrates that GPT-4 has the potential to transform discharge letters into more patient-centered communication. While the readability and patient centricity of the transformed letters are well-established, they do not fully address all patient safety–relevant information, resulting in the omission of key aspects. Further optimization of prompt engineering may help address this issue and improve the completeness of the transformation.

2 December 2021 in JMIR mHealth and uHealth
Verified
Journal Article
Open access
AHAnne Herrmann-Werner
Anne Herrmann-Werner
TLTeresa Loda
Teresa Loda
Stephan ZipfelStephan Zipfel
Stephan Zipfel
MHMartin Holderried
Martin Holderried
FHFriederike Holderried
Friederike Holderried
RERebecca Erschens
Rebecca Erschens
Background: Language barriers in medical encounters pose risks for interactions with patients, their care, and their outcomes. Because human translators, the gold standard for mitigating language barriers, can be cost- and time-intensive, mechanical alternatives such as language translation apps (LTA) have gained in popularity. However, adequate training for physicians in using LTAs remains elusive. Objective: A proof-of-concept pilot study was designed to evaluate the use of a speech-to-speech LTA in a specific simulated physician-patient situation, particularly its perceived usability, helpfulness, and meaningfulness, and to assess the teaching unit overall. Methods: Students engaged in a 90-min simulation with a standardized patient (SP) and the LTA iTranslate Converse. Thereafter, they rated the LTA with six items—helpful, intuitive, informative, accurate, recommendable, and applicable—on a 7-point Likert scale ranging from 1 (don’t agree at all) to 7 (completely agree) and could provide free-text responses for four items: general impression of the LTA, the LTA’s benefits, the LTA’s risks, and suggestions for improvement. Students also assessed the teaching unit on a 6-point scale from 1 (excellent) to 6 (insufficient). Data were evaluated quantitatively with mean (SD) values and qualitatively in thematic content analysis. Results: Of 111 students in the course, 76 (68.5%) participated (59.2% women, age 20.7 years, SD 3.3 years). Values for the LTA’s being helpful (mean 3.45, SD 1.79), recommendable (mean 3.33, SD 1.65) and applicable (mean 3.57, SD 1.85) were centered around the average of 3.5. The items intuitive (mean 4.57, SD 1.74) and informative (mean 4.53, SD 1.95) were above average. The only below-average item concerned its accuracy (mean 2.38, SD 1.36). Students rated the teaching unit as being excellent (mean 1.2, SD 0.54) but wanted practical training with an SP plus a simulated human translator first. Free-text responses revealed several concerns about translation errors that could jeopardize diagnostic decisions. Students feared that patient-physician communication mediated by the LTA could decrease empathy and raised concerns regarding data protection and technical reliability. Nevertheless, they appreciated the LTA’s cost-effectiveness and usefulness as the best option when the gold standard is unavailable. They also reported wanting more medical-specific vocabulary and images to convey all information necessary for medical communication. Conclusions: This study revealed the feasibility of using a speech-to-speech LTA in an undergraduate medical course. Although human translators remain the gold standard, LTAs could be valuable alternatives. Students appreciated the simulated teaching and recognized the LTA’s potential benefits and risks for use in real-world clinical settings. To optimize patients’ and health care professionals’ experiences with LTAs, future investigations should examine specific design options for training interventions and consider the legal aspects of human-machine interaction in health care settings.

1 June 2018 in Telemedicine and e-Health
Verified
Journal Article
Closed access
MHMartin Holderried
Martin Holderried
AHAnsgar Hoeper
Ansgar Hoeper
FHFriederike Holderried
Friederike Holderried
GBGunnar Blumenstock
Gunnar Blumenstock
CEChristian Ernst
Christian Ernst
ATAnke Tropitzsch
Anke Tropitzsch
Introduction: Online communication and the number of e-health applications have noticeably increased. However, little is known about the otolaryngologists' use behavior and their attitudes toward the potential of e-health. The aims of the study were to evaluate the documentation, information, and communication technologies used by otolaryngologists and to get a better understanding of their attitudes toward the potential of e-health for cross-sectoral patient care. Methods: A survey was developed and tested by otolaryngologists, healthcare-information technology experts, and health services researchers. A total of 334 otolaryngologists in private practice were asked to participate in this cross-sectional study. In total, 234 of them took part in the study, and 157 returned completed questionnaires. Statistical analysis was performed by using crosstabs, including chi-square tests, and multivariate logistic regressions. Results and Materials: Digital technologies are widely used by otolaryngologists (e.g., 89.6% use an electronic health record). However, the majority of intersectoral communication is still based on analogue techniques (e.g., fax use in 63.7%). From the otolaryngologists' perspectives, the potential of e-health for intersectoral care is mostly in appointment scheduling, further referrals to hospitals, and automated appointment reminders. The physicians' attitudes toward e-health are associated with their Internet use behavior in daily life (odds ratio = 4.30, confidence interval 1.11–16.64, p = 0.035) but not with their demographics. Discussion: The otolaryngologists are well prepared and have an overall positive attitude toward e-health for deeper use in cross-sectoral care. Therefore, e-health in otolaryngology needs more attention and resources for further studies, especially with a focus on quality and safety of care.