Objective: This study aimed to develop and evaluate an artificial intelligence (AI)-driven system for forecasting Pediatric Emergency Department (PED) overcrowding and optimizing physician shift schedules using machine learning operations (MLOps). Materials and Methods: Data from 352 843 PED admissions between January 2018 and May 2023 were analyzed. Twenty time-series forecasting models—including classical methods and advanced deep learning architectures like Temporal Convolutional Network, Time-series Dense Encoder and Reversible Instance Normalization, Neural High-order Time Series model, and Neural Basis Expansion Analysis—were developed and compared using Python 3.8. Starting in January 2023, an MLOps simulation automated data updates and model retraining. Shift schedules were optimized based on forecasted patient volumes using integer linear programming. Results: Advanced deep learning models outperformed traditional models, achieving initial R2 scores up to 75%. Throughout the simulation, the median R2 score for all models was 44% after MLOps-based model selection, the median R2 improved to 60%. The MLOps architecture facilitated continuous model updates, enhancing forecast accuracy. Shift optimization adjusted staffing in 69 out of 84 shifts, increasing physician allocation by up to 30.4% during peak hours. This adjustment reduced the patient-to-physician ratio by an average of 4.32 patients during the 8-16 shift and 4.40 patients during the 16-24 shift. Discussion: The integration of advanced deep learning models with MLOps architecture allowed for continuous model updates, enhancing the accuracy of PED overcrowding forecasts and outperforming traditional methods. The AI-driven system demonstrated resilience against data drift caused by events like the COVID-19 pandemic, adapting to changing conditions. Optimizing physician shifts based on these forecasts improved workforce distribution without increasing staff numbers, reducing patient load per physician during peak hours. However, limitations include the single-center design and a fixed staffing model, indicating the need for multicenter validation and implementation in settings with dynamic staffing practices. Future research should focus on expanding datasets through multicenter collaborations and developing forecasting models that provide longer lead times without compromising accuracy. Conclusions: The AI-driven forecasting and shift optimization system demonstrated the efficacy of integrating AI and MLOps in predicting PED overcrowding and optimizing physician shifts. This approach outperformed traditional methods, highlighting its potential for managing overcrowding in emergency departments. Future research should focus on multicenter validation and real-world implementation to fully leverage the benefits of this innovative system.
31 March 2025 in BMC Medical Informatics and Decision Making
The integration of big data and artificial intelligence (AI) in healthcare, particularly through the analysis of electronic health records (EHR), presents significant opportunities for improving diagnostic accuracy and patient outcomes. However, the challenge of processing and accurately labeling vast amounts of unstructured data remains a critical bottleneck, necessitating efficient and reliable solutions. This study investigates the ability of domain specific, fine-tuned large language models (LLMs) to classify unstructured EHR texts with typographical errors through named entity recognition tasks, aiming to improve the efficiency and reliability of supervised learning AI models in healthcare. Turkish clinical notes from pediatric emergency room admissions at Hacettepe University İhsan Doğramacı Children’s Hospital from 2018 to 2023 were analyzed. The data were preprocessed with open source Python libraries and categorized using a pretrained GPT-3 model, “text-davinci-003,” before and after fine-tuning with domain-specific data on respiratory tract infections (RTI). The model’s predictions were compared against ground truth labels established by pediatric specialists. Out of 24,229 patient records classified as poorly labeled, 18,879 were identified without typographical errors and confirmed for RTI through filtering methods. The fine-tuned model achieved a 99.88% accuracy, significantly outperforming the pretrained model’s 78.54% accuracy in identifying RTI cases among the remaining records. The fine-tuned model demonstrated superior performance metrics across all evaluated aspects compared to the pretrained model. Fine-tuned LLMs can categorize unstructured EHR data with high accuracy, closely approximating the performance of domain experts. This approach significantly reduces the time and costs associated with manual data labeling, demonstrating the potential to streamline the processing of large-scale healthcare data for AI applications.