Machine learning (ML) has been used in different ways in the fight against COVID-19 disease. ML models have been developed, e.g., for diagnostic or prognostic purposes and using various modalities of data (e.g., textual, visual, or structured). Due to the many specific aspects of this disease and its evolution over time, there is still not enough understanding of all relevant factors influencing the course of COVID-19 in particular patients. In all aspects of our work, there was a strong involvement of a medical expert following the human-in-the-loop principle. This is a very important but usually neglected part of the ML and knowledge extraction (KE) process. Our research shows that explainable artificial intelligence (XAI) may significantly support this part of ML and KE. Our research focused on using ML for knowledge extraction in two specific scenarios. In the first scenario, we aimed to discover whether adding information about the predominant COVID-19 variant impacts the performance of the ML models. In the second scenario, we focused on prognostic classification models concerning the need for an intensive care unit for a given patient in connection with different explainability AI (XAI) methods. We have used nine ML algorithms, namely XGBoost, CatBoost, LightGBM, logistic regression, Naive Bayes, random forest, SGD, SVM-linear, and SVM-RBF. We measured the performance of the resulting models using precision, accuracy, and AUC metrics. Subsequently, we focused on knowledge extraction from the best-performing models using two different approaches as follows: (a) features extracted automatically by forward stepwise selection (FSS); (b) attributes and their interactions discovered by model explainability methods. Both were compared with the attributes selected by the medical experts in advance based on the domain expertise. Our experiments showed that adding information about the COVID-19 variant did not influence the performance of the resulting ML models. It also turned out that medical experts were much more precise in the identification of significant attributes than FSS. Explainability methods identified almost the same attributes as a medical expert and interesting interactions among them, which the expert discussed from a medical point of view. The results of our research and their consequences are discussed.
Interest in explainable machine learning has grown, particularly in healthcare, where transparency and trust are essential. We developed a semi-automated evaluation framework within a clinical decision support system (CDSS-EQCM) that integrates LIME and SHAP explanations with multi-criteria decision-making (TOPSIS and Borda count) to rank model interpretability. After two-phase preprocessing of 2934 COVID-19 patient records spanning four epidemic waves, we applied five classifiers (Random Forest, Decision Tree, Logistic Regression, k-NN, SVM). Five infectious disease physicians used a Streamlit interface to generate patient-specific explanations and rate models on accuracy, separability, stability, response time, understandability, and user experience. Random Forest combined with SHAP consistently achieved the highest rankings in Borda count. Clinicians reported reduced evaluation time, enhanced explanation clarity, and increased confidence in model outputs. These results demonstrate that CDSS-EQCM can effectively streamline interpretability assessment and support clinician decision-making in medical diagnostics. Future work will focus on deeper electronic medical record integration and interactive parameter tuning to further enhance real-time diagnostic support.