Interpretability is just as important as accuracy when it comes to complex models, especially in the context of deep learning models. Explainable artificial intelligence (XAI) approaches have been developed to address this problem. The literature on XAI for spectroscopy mainly emphasizes independent feature analysis with limited application of zone analysis. Individual feature analysis methods, such as shapley additive explanations (SHAP) and local interpretable model-agnostic explanations (LIME), have limitations due to their dependence on perturbations. These methods measure how AI models respond to sudden changes in the individual feature values. While they can help identify the most impactful features, the abrupt shifts introduced by replacing these values with zero or the expected ones may not accurately represent real-world scenarios. This can lead to mathematical and computational interpretations that are neither physically realistic nor intuitive to humans. Our proposed method does not rely on individual disturbances. Instead, it targets "spectral zones" to directly estimate the effect of group disturbances on a trained model. Consequently, factors such as sample size, hyperparameter selection, and other training-related considerations are not the primary focus of the XAI methods. To achieve this, we have developed a modified version of LIME and SHAP capable of performing group perturbations, enhancing explainability and realism while minimizing noise in the plots used for interpretability. Additionally, we employed an efficient approach to calculate spectral zones for complex spectra with indistinct spectral boundaries. Users can also define the zones themselves using their domain-specific knowledge.

The rising incidence of head and neck cancer represents a serious global health challenge, requiring more accurate diagnosis and innovative surgical approaches. Multimodal nonlinear optical microscopy, combining coherent anti-Stokes Raman scattering (CARS), two-photon excited fluorescence (TPEF), and second-harmonic generation (SHG) with deep learning–based analysis routines, offers label-free assessment of the tissue’s morphochemical composition and allows early-stage and automatic detection of disease. For clinical intraoperative application, compact devices are required. In this preclinical study, a cohort of 15 patients was examined with a newly developed rigid CARS/TPEF/SHG endomicroscope. To detect head and neck tumor from the multimodal data, deep learning–based semantic segmentation models were used. This preclinical study yields in a diagnostic sensitivity of 88% and a specificity of 96%. To combine diagnostics with therapy, machine learning–inspired image-guided selective tissue removal was used by integrating femtosecond laser ablation into the endomicroscope. This enables a powerful approach of intraoperative “seek and treat,” paving the way to advanced surgical treatment.

Identifying bacterial strains is essential in microbiology for various practical applications, such as disease diagnosis and quality monitoring of food and water. Classical machine learning algorithms have been utilized to identify bacteria based on their Raman spectra. However, convolutional neural networks (CNNs) offer higher classification accuracy, but they require extensive training sets and retraining of previous untrained class targets can be costly and time-consuming. Siamese networks have emerged as a promising solution. They are composed of two CNNs with the same structure and a final network that acts as a distance metric, converting the classification problem into a similarity problem. Classical machine learning approaches, shallow and deep CNNs, and two Siamese network variants were tailored and tested on Raman spectral datasets of bacteria. The methods were evaluated based on mean sensitivity, training time, prediction time, and the number of parameters. In this comparison, Siamese-model2 achieved the highest mean sensitivity of 83.61 ± 4.73 and demonstrated remarkable performance in handling unbalanced and limited data scenarios, achieving a prediction accuracy of 73%. Therefore, the choice of model depends on the specific trade-off between accuracy, (prediction/training) time, and resources for the particular application. Classical machine learning models and shallow CNN models may be more suitable if time and computational resources are a concern. Siamese networks are a good choice for small datasets and CNN for extensive data.

Machine learning algorithms for bacterial strain identification using Raman spectroscopy have been widely used in microbiology. During the training phase, existing datasets are augmented and used to optimize model architecture and hyperparameters. After training, it is presumed that the models have reached their peak performance and are used for inference without being further enhanced. Our methodology combines Monte Carlo Dropout (MCD) with convolutional neural networks (CNNs) by utilizing dropout during the inference phase, which enables to measure the model uncertainty, a critical but often ignored aspect in deep learning models. We categorize unseen input data into two subsets based on the uncertainty of their prediction by employing MCD and defining the threshold using the Gaussian Mixture Model (GMM). The final prediction is obtained on the subset of testing data that exhibits lower model uncertainty, thereby enhancing the reliability of the results. To validate our method, we applied it to two Raman spectra datasets. As a result, we have observed an increase in accuracy of 9 % for Dataset 1 (from 83.10 % to 92.10 %) and 12.82 % for Dataset 2 (from 83.86 % to 96.68 %). These improvements were observed within specific subsets of the data: 826 out of 1206 spectra in Dataset 1 and 1700 out of 3000 spectra in Dataset 2. This demonstrates the effectiveness of our approach in improving prediction accuracy by focusing on data with lower uncertainty. Different from routine prediction based on mere probabilities, we believe this uncertainty-guided prediction is more effective to ensure a high prediction rate rather than the prediction on the entire dataset. By guiding the decision-making of a model on higher-confidence subsets, our methodology can enhance the accuracy of classification in critical areas like disease diagnosis and safety monitoring. This targeted approach is to advance microbial identification and produces more trustworthy predictions.

Deep learning (DL) has significantly advanced Raman spectra analysis, achieving high accuracy and efficiency. However, their complexity and opacity limit their application in areas where understanding and transparency are essential. To address this, we present XAI‐2DCOS, an innovative eXplainable Artificial Intelligence (XAI) framework that employs 2D correlation spectroscopy (2DCOS). Traditionally, 2DCOS reveals the sequence of molecular changes under varying conditions. We repurpose it to enhance the interpretability of DL models by linking changes in spectral features to model outputs, identifying critical wavenumbers, and how their variations affect model accuracy. We applied XAI‐2DCOS to a DL model trained on a dataset of oil Raman spectra, demonstrating its ability to identify critical spectral features that align with domain knowledge. To improve robustness, we integrated a conditional generative adversarial network (CGAN) for data augmentation. CGAN generates synthetic data, ensuring the presence of spectra across the entire probability range. A normalized relevance score quantifies the contribution for each wavenumber to the model's prediction. A predictive probability map delineates decision boundaries within the PCA space. Synchronous 2DCOS maps are used to guide spectral adjustments that improve prediction confidence for specific class predictions. These adjustments can affect multiple output classes with differential scaling of output activations, suggesting that crossing a threshold can shift the model decision. Our results show that XAI‐2DCOS improves the interpretability and reliability of DL models applied to Raman spectra. Furthermore, CGAN data augmentation extends the applicability of XAI‐2DCOS to smaller datasets.

point clouds with high spatial resolution. For such data with huge amounts of points, the classification of each individual 3D point is an intractable task. Instead, we propose to segment the scene into meaningful regions as a first step. Afterward, we classify these segments using a combination of PointNet and geometric deep learning. This two-step approach resembles object-based image analysis. As an additional novelty, we apply surface normalization techniques and enrich features with geometric attributes. Our experiments show the potential of this approach for a variety of outdoor scene analysis tasks. In particular, we are able to reach 89.6% overall accuracy and 64.4% average intersection over union (IoU) in the Semantic3D benchmark. Furthermore, we achieve 66.7% average IoU on Paris-Lille-3D. We also successfully apply our approach to the automatic semantic analysis of forestry data.