Wildfires stand as one of the most relevant natural disasters worldwide, particularly more so due to the effect of climate change and its impact on various societal and environmental levels. In this regard, a significant amount of research has been done in order to address this issue, deploying a wide variety of technologies and following a multi-disciplinary approach. Notably, computer vision has played a fundamental role in this regard. It can be used to extract and combine information from several imaging modalities in regard to fire detection, characterization and wildfire spread forecasting. In recent years, there has been work pertaining to Deep Learning (DL)-based fire segmentation, showing very promising results. However, it is currently unclear whether the architecture of a model, its loss function, or the image type employed (visible, infrared, or fused) has the most impact on the fire segmentation results. In the present work, we evaluate different combinations of state-of-the-art (SOTA) DL architectures, loss functions, and types of images to identify the parameters most relevant to improve the segmentation results. We benchmark them to identify the top-performing ones and compare them to traditional fire segmentation techniques. Finally, we evaluate if the addition of attention modules on the best performing architecture can further improve the segmentation results. To the best of our knowledge, this is the first work that evaluates the impact of the architecture, loss function, and image type in the performance of DL-based wildfire segmentation models.

Early wildfire detection is of paramount importance to avoid as much damage as possible to the environment, properties, and lives. Deep Learning (DL) models that can leverage both visible and infrared information have the potential to display state-of-the-art performance, with lower false-positive rates than existing techniques. However, most DL-based image fusion methods have not been evaluated in the domain of fire imagery. Additionally, to the best of our knowledge, no publicly available dataset contains visible-infrared fused fire images. There is a growing interest in DL-based image fusion techniques due to their reduced complexity. Due to the latter, we select three state-of-the-art, DL-based image fusion techniques and evaluate them for the specific task of fire image fusion. We compare the performance of these methods on selected metrics. Finally, we also present an extension to one of the said methods, that we called FIRe-GAN, that improves the generation of artificial infrared images and fused ones on selected metrics.

Deep learning models for analyzing 3D point clouds have been extensively researched. However, there is limited research on explainable artificial intelligence techniques for such models. Approaches relying on saliency maps aim to attribute model outputs to their input. Nevertheless, their practical use for extracting meaningful insights on model behavior depends on user knowledge, available visualization tools, and ad hoc data analytics pipelines. In this work, we propose a system that combines a natural language interface with a 3D point cloud visualization tool for visualizing and analyzing 3D point clouds and their corresponding saliency maps. Our system can 1) answer general questions in the context of 3D point cloud semantic segmentation and saliency maps, 2) answer data analytics-related questions about the 3D point cloud and its associated saliency map, and 3) generate specifications for a high-level grammar for interactive graphics to generate visualizations that aid in the extraction of meaningful insights. We increase the transparency of the answers by providing users with the sources from which the large language model draws its answers and any generated code used for data analytics. We demonstrate the proposed system’s workflow through a use case study involving a sample input to a 3D point cloud semantic segmentation model, and evaluate the reliability of the natural language model outputs. Our results indicate that including a natural language interface facilitates the exploration of the data and enhances the efficiency of the data analytics process, enhancing the value provided by the 3D point cloud saliency maps.

Deep learning models achieve high accuracy in the semantic segmentation of 3D point clouds. However, it is challenging to discern which patterns a model has learned and how it derives its output from the input. In order to explain semantic segmentation models for 3D point clouds, saliency techniques can be applied to them. These methods generate saliency maps that visualize the contribution of input points to a particular model output. However, there is a lack of quantitative evaluation regarding the reliability of the generated saliency maps and the influence of different hyperparameters of the saliency techniques on the methods’ results. In this paper, we quantitatively evaluate the reliability of saliency maps generated by Integrated Gradients and pGS-CAM. To this end, we adapt established sanity checks from the image domain to 3D point cloud semantic segmentation. Our results suggest partial sensitivity to the parameters of the model and training labels for both saliency techniques. Furthermore, Integrated Gradients shows instability concerning the choice of baseline. Using ablation tests that gradually remove points with high or low attribution values, we show that pGS-CAM can reliably identify points with high and low contributions to the model output and that the performance of Integrated Gradients is heavily dependent on the baseline choice. Finally, we propose an averaging approach to aggregate the results of points that receive multiple saliency scores during the segmentation process and show that it produces saliency maps that better reflect high-contribution input points than previous approaches used for Integrated Gradients.