Large Language Models (LLMs) are increasingly transforming medical applications. However, proprietary models such as GPT-4o face significant barriers to clinical adoption because they cannot be deployed on site within healthcare institutions, making them non-compliant with stringent privacy regulations. Recent advancements in open-source LLMs such as DeepSeek models offer a promising alternative since they allow efficient fine-tuning on local data in hospitals with advanced IT infrastructure. To demonstrate the clinical utility of DeepSeek-V3 and DeepSeek-R1, we benchmarked their performance on clinical decision support tasks against proprietary LLMs, including GPT-4o and Gemini-2.0 Flash Thinking Experimental. Using 125 patient cases with sufficient statistical power, covering a broad range of frequent and rare diseases, we found that DeepSeek models perform equally well and in some cases better than proprietary LLMs. Our study demonstrates that open-source LLMs can provide a scalable pathway for secure model training enabling real-world medical applications in accordance with data privacy and healthcare regulations.
It is likely that individuals are turning to Large Language Models (LLMs) to seek health advice, much like searching for diagnoses on Google. We evaluate clinical accuracy of GPT-3·5 and GPT-4 for suggesting initial diagnosis, examination steps and treatment of 110 medical cases across diverse clinical disciplines. Moreover, two model configurations of the Llama 2 open source LLMs are assessed in a sub-study. For benchmarking the diagnostic task, we conduct a naïve Google search for comparison. Overall, GPT-4 performed best with superior performances over GPT-3·5 considering diagnosis and examination and superior performance over Google for diagnosis. Except for treatment, better performance on frequent vs rare diseases is evident for all three approaches. The sub-study indicates slightly lower performances for Llama models. In conclusion, the commercial LLMs show growing potential for medical question answering in two successive major releases. However, some weaknesses underscore the need for robust and regulated AI models in health care. Open source LLMs can be a viable option to address specific needs regarding data privacy and transparency of training.
Valid variant calling results are crucial for the use of next-generation sequencing in clinical routine. However, there are numerous variant calling tools that usually differ in algorithms, filtering strategies, recommendations and thus, also in the output. We evaluated eight open-source tools regarding their ability to call single nucleotide variants and short indels with allelic frequencies as low as 1% in non-matched next-generation sequencing data: GATK HaplotypeCaller, Platypus, VarScan, LoFreq, FreeBayes, SNVer, SAMtools and VarDict. We analysed two real datasets from patients with myelodysplastic syndrome, covering 54 Illumina HiSeq samples and 111 Illumina NextSeq samples. Mutations were validated by re-sequencing on the same platform, on a different platform and expert based review. In addition we considered two simulated datasets with varying coverage and error profiles, covering 50 samples each. In all cases an identical target region consisting of 19 genes (42,322 bp) was analysed. Altogether, no tool succeeded in calling all mutations. High sensitivity was always accompanied by low precision. Influence of varying coverages- and background noise on variant calling was generally low. Taking everything into account, VarDict performed best. However, our results indicate that there is a need to improve reproducibility of the results in the context of multithreading.
Cancer development is a dynamic process during which the successive accumulation of mutations results in cells with increasingly malignant characteristics. Here, we show the clonal evolution pattern in myelodysplastic syndrome (MDS) patients receiving supportive care, with or without lenalidomide (follow-up 2.5–11 years). Whole-exome and targeted deep sequencing at multiple time points during the disease course reveals that both linear and branched evolutionary patterns occur with and without disease-modifying treatment. The application of disease-modifying therapy may create an evolutionary bottleneck after which more complex MDS, but also unrelated clones of haematopoietic cells, may emerge. In addition, subclones that acquired an additional mutation associated with treatment resistance (TP53) or disease progression (NRAS, KRAS) may be detected months before clinical changes become apparent. Monitoring the genetic landscape during the disease may help to guide treatment decisions.
23 September 2017 in Journal of the American Medical Informatics Association
To systematically classify the clinical impact of computerized clinical decision support systems (CDSSs) in inpatient care. Medline, Cochrane Trials, and Cochrane Reviews were searched for CDSS studies that assessed patient outcomes in inpatient settings. For each study, 2 physicians independently mapped patient outcome effects to a predefined medical effect score to assess the clinical impact of reported outcome effects. Disagreements were measured by using weighted kappa and solved by consensus. An example set of promising disease entities was generated based on medical effect scores and risk of bias assessment. To summarize technical characteristics of the systems, reported input variables and algorithm types were extracted as well. Seventy studies were included. Five (7%) reported reduced mortality, 16 (23%) reduced life-threatening events, and 28 (40%) reduced non–life-threatening events, 20 (29%) had no significant impact on patient outcomes, and 1 showed a negative effect (weighted κ: 0.72, P < .001). Six of 24 disease entity settings showed high effect scores with medium or low risk of bias: blood glucose management, blood transfusion management, physiologic deterioration prevention, pressure ulcer prevention, acute kidney injury prevention, and venous thromboembolism prophylaxis. Most of the implemented algorithms (72%) were rule-based. Reported input variables are shared as standardized models on a metadata repository. Most of the included CDSS studies were associated with positive patient outcomes effects but with substantial differences regarding the clinical impact. A subset of 6 disease entities could be filtered in which CDSS should be given special consideration at sites where computer-assisted decision-making is deemed to be underutilized. on PROSPERO: CRD42016049946.
While survival has improved for Burkitt lymphoma patients, potential differences in outcome between pediatric and adult patients remain unclear. In both age groups, survival remains poor at relapse. Therefore, we conducted a comparative study in a large pediatric cohort, including 191 cases and 97 samples from adults. WhileTP53andCCND3mutation frequencies are not age related, samples from pediatric patients showed a higher frequency of mutations inID3,DDX3X, ARID1AandSMARCA4, while several genes such asBCL2andYY1AP1are almost exclusively mutated in adult patients. An unbiased analysis reveals a transition of the mutational profile between 25 and 40 years of age. Survival analysis in the pediatric cohort confirms thatTP53mutations are significantly associated with higher incidence of relapse (25 ± 4% versus 6 ± 2%, p-value 0.0002). This identifies a promising molecular marker for relapse incidence in pediatric BL which will be used in future clinical trials.