Yoon M, et al. Int J Heart Fail. 2024 Jan;6(1):11-19. https://doi.org/10.36628/ijhf.2023.0050

Background: Adherence to oral anticoagulant therapy is essential to prevent ischemic stroke in patients with atrial fibrillation (AF). Objective: This study aimed to evaluate whether smartphone app–based interventions improve medication adherence in patients with AF. Methods: This open-label, multicenter randomized controlled trial (ADHERE-App [Self-Awareness of Drug Adherence to Edoxaban Using an Automatic App Feedback System] study) enrolled patients with AF treated with edoxaban for stroke prevention. They were randomly assigned to app-conditioned feedback (intervention; n=248) and conventional treatment (control; n=250) groups. The intervention group received daily alerts via a smartphone app to take edoxaban and measure blood pressure and heart rate at specific times. The control group received only standard, guideline-recommended care. The primary end point was edoxaban adherence, measured by pill count at 3 or 6 months. Medication adherence and the proportion of adequate medication adherence, which was defined as ≥95% of continuous medication adherence, were evaluated. Results: Medication adherence at 3 or 6 months was not significantly different between the intervention and control groups (median 98%, IQR 95%-100% vs median 98%, IQR 91%-100% at 3 months, P=.06; median 98%, IQR 94.5%-100% vs median 97.5%, IQR 92.8%-100% at 6 months, P=.15). However, the proportion of adequate medication adherence (≥95%) was significantly higher in the intervention group at both time points (76.8% vs 64.7% at 3 months, P=.01; 73.9% vs 61% at 6 months, P=.007). Among patients aged >65 years, the intervention group showed a higher medication adherence value and a higher proportion of adequate medication adherence (≥95%) at 6 months. Conclusions: There was no difference in edoxaban adherence between the groups. However, the proportion of adequate medication adherence was higher in the intervention group, and the benefit of the smartphone app–based intervention on medication adherence was more pronounced among older patients than among younger patients. Given the low adherence to oral anticoagulants, especially among older adults, using a smartphone app may potentially improve medication adherence. Trial Registration: International Clinical Trials Registry Platform KCT0004754; https://cris.nih.go.kr/cris/search/detailSearch.do?seq=28496&search_page=L International Registered Report Identifier (IRRID): RR2-10.1136/bmjopen-2021-048777

Automated medical diagnosis is one of the important machine learning applications in the domain of healthcare. In this regard, most of the approaches primarily focus on optimizing the accuracy of classification models. In this research, we argue that, unlike general-purpose classification problems, medical applications, such as chronic kidney disease (CKD) diagnosis, require special treatment. In the case of CKD, apart from model performance, other factors such as the cost of data acquisition may also be taken into account to enhance the applicability of the automated diagnosis system. In this research, we proposed two techniques for cost-sensitive feature ranking. An ensemble of decision tree models is employed in both the techniques for computing the worth of a feature in the CKD dataset. An automatic threshold selection heuristic is also introduced which is based on the intersection of features’ worth and their accumulated cost. A set of experiments are conducted to evaluate the efficacy of the proposed techniques on both tree-based and non tree-based classification models. The proposed approaches were also evaluated against several comparative techniques. Furthermore, it is demonstrated that the proposed techniques select around 1/4th of the original CKD features while reducing the cost by a factor of 7.42 of the original feature set. Based on the extensive experimentation, it is concluded that the proposed techniques employing feature-cost interaction heuristic tend to select feature subsets that are both useful and cost-effective.

Background: People with schizophrenia often present with cognitive impairments that may hinder their ability to learn about their condition. Education platforms powered by large language models (LLMs) have the potential to improve the accessibility of mental health information. However, the black-box nature of LLMs raises ethical and safety concerns regarding the controllability of chatbots. In particular, prompt-engineered chatbots may drift from their intended role as the conversation progresses and become more prone to hallucinations. Objective: This study aimed to develop and evaluate a critical analysis filter (CAF) system that ensures that an LLM-powered prompt-engineered chatbot reliably complies with its predefined instructions and scope while delivering validated mental health information. Methods: For a proof of concept, we prompt engineered an educational chatbot for schizophrenia powered by GPT-4 that could dynamically access information from a schizophrenia manual written for people with schizophrenia and their caregivers. In the CAF, a team of prompt-engineered LLM agents was used to critically analyze and refine the chatbot’s responses and deliver real-time feedback to the chatbot. To assess the ability of the CAF to re-establish the chatbot’s adherence to its instructions, we generated 3 conversations (by conversing with the chatbot with the CAF disabled) wherein the chatbot started to drift from its instructions toward various unintended roles. We used these checkpoint conversations to initialize automated conversations between the chatbot and adversarial chatbots designed to entice it toward unintended roles. Conversations were repeatedly sampled with the CAF enabled and disabled. In total, 3 human raters independently rated each chatbot response according to criteria developed to measure the chatbot’s integrity, specifically, its transparency (such as admitting when a statement lacked explicit support from its scripted sources) and its tendency to faithfully convey the scripted information in the schizophrenia manual. Results: In total, 36 responses (3 different checkpoint conversations, 3 conversations per checkpoint, and 4 adversarial queries per conversation) were rated for compliance with the CAF enabled and disabled. Activating the CAF resulted in a compliance score that was considered acceptable (≥2) in 81% (7/36) of the responses, compared to only 8.3% (3/36) when the CAF was deactivated. Conclusions: Although more rigorous testing in realistic scenarios is needed, our results suggest that self-reflection mechanisms could enable LLMs to be used effectively and safely in educational mental health platforms. This approach harnesses the flexibility of LLMs while reliably constraining their scope to appropriate and accurate interactions.

Clinical Practice Guidelines (CPGs) aim to optimize patient care by assisting physicians during the decision-making process. However, guideline adherence is highly affected by its unstructured format and aggregation of background information with disease-specific information. The objective of our study is to extract disease-specific information from CPG for enhancing its adherence ratio. In this research, we propose a semi-automatic mechanism for extracting disease-specific information from CPGs using pattern-matching techniques. We apply supervised and unsupervised machine-learning algorithms on CPG to extract a list of salient terms contributing to distinguishing recommendation sentences (RS) from non-recommendation sentences (NRS). Simultaneously, a group of experts also analyzes the same CPG and extract the initial patterns “Heuristic Patterns” using a group decision-making method, nominal group technique (NGT). We provide the list of salient terms to the experts and ask them to refine their extracted patterns. The experts refine patterns considering the provided salient terms. The extracted heuristic patterns depend on specific terms and suffer from the specialization problem due to synonymy and polysemy. Therefore, we generalize the heuristic patterns to part-of-speech (POS) patterns and unified medical language system (UMLS) patterns, which make the proposed method generalize for all types of CPGs. We evaluated the initial extracted patterns on asthma, rhinosinusitis, and hypertension guidelines with the accuracy of 76.92%, 84.63%, and 89.16%, respectively. The accuracy increased to 78.89%, 85.32%, and 92.07% with refined machine-learning assistive patterns, respectively. Our system assists physicians by locating disease-specific information in the CPGs, which enhances the physicians’ performance and reduces CPG processing time. Additionally, it is beneficial in CPGs content annotation.