Forest fires have the characteristics of strong unpredictability and extreme destruction. Hence, it is difficult to carry out effective prevention and control. Once the fire spreads, devastating damage will be caused to natural resources and the ecological environment. In order to detect early forest fires in real-time and provide firefighting assistance, we propose a vision-based detection and spatial localization scheme and develop a system carried on the unmanned aerial vehicle (UAV) with an OAK-D camera. During the high incidence of forest fires, UAVs equipped with our system are deployed to patrol the forest. Our scheme includes two key aspects. First, the lightweight model, NanoDet, is applied as a detector to identify and locate fires in the vision field. Techniques such as the cosine learning rate strategy and data augmentations are employed to further enhance mean average precision (mAP). After capturing 2D images with fires from the detector, the binocular stereo vision is applied to calculate the depth map, where the HSV-Mask filter and non-zero mean method are proposed to eliminate the interference values when calculating the depth of the fire area. Second, to get the latitude, longitude, and altitude (LLA) coordinates of the fire area, coordinate frame conversion is used along with data from the GPS module and inertial measurement unit (IMU) module. As a result, we experiment with simulated fire in a forest area to test the effectiveness of this system. The results show that 89.34% of the suspicious frames with flame targets are detected and the localization error of latitude and longitude is in the order of 10⁻⁵ degrees; this demonstrates that the system meets our precision requirements and is sufficient for forest fire inspection.

Forest fires are destructive and rapidly spreading, causing great harm to forest ecosystems and humans. Deep learning techniques can adaptively learn and extract features of forest fires and smoke. However, the complex backgrounds and different forest fire and smoke features in captured forest fire images make detection difficult. Facing the complex background of forest fire smoke, it is difficult for traditional machine learning methods to design a general feature extraction module for feature extraction. Deep learning methods are effective in many fields, so this paper improves on the You Only Look Once v5 (YOLOv5s) model, and the improved model has better detection performance for forest fires and smoke. First, a coordinate attention (CA) model is integrated into the YOLOv5 model to highlight fire smoke targets and improve the identifiability of different smoke features. Second, we replaced YOLOv5s original spatial pyramidal ensemble fast (SPPF) module with a receptive field block (RFB) module to enable better focus on the global information of different fires. Third, the path aggregation network (PANet) of the neck structure in the YOLOv5s model is improved to a bi-directional feature pyramid network (Bi-FPN). Compared with the YOLOv5 model, our improved forest fire and smoke detection model at mAP@0.5 improves by 5.1%.

Deep learning-based forest fire vision monitoring methods have developed rapidly and are becoming mainstream. The existing methods, however, are based on enormous amounts of data, and have issues with weak feature extraction, poor small target recognition and many missed and false detections in complex forest scenes. In order to solve these problems, we proposed a multi-task learning-based forest fire detection model (MTL-FFDet), which contains three tasks (the detection task, the segmentation task and the classification task) and shares the feature extraction module. In addition, to improve detection accuracy and decrease missed and false detections, we proposed the joint multi-task non-maximum suppression (NMS) processing algorithm that fully utilizes the advantages of each task. Furthermore, considering the objective fact that divided flame targets in an image are still flame targets, our proposed data augmentation strategy of a diagonal swap of random origin is a good remedy for the poor detection effect caused by small fire targets. Experiments showed that our model outperforms YOLOv5-s in terms of $mAP$ (mean average precision) by 3.2%, $A{P}_S$ (average precision for small objects) by 4.8%, $A{R}_S$ (average recall for small objects) by 4.0%, and other metrics by 1% to 2%. Finally, the visualization analysis showed that our multi-task model can focus on the target region better than the single-task model during feature extraction, with superior extraction ability.

Forest fires cause devastating damage to the natural environment, making prompt and precise detection of smoke and fires in forests crucial. When processing forest fire images based on ground and aerial perspectives, current object detection methods still encounter issues, such as inadequate detection precision, elevated false detection and omission rates, as well as difficulties in detecting small targets in complex forest environments. Multi-task learning represents a framework in machine learning where a model can handle detection and segmentation tasks concurrently, enhancing the accuracy and generalization capacity for object detection. Therefore, this study proposes a Multi-Task Learning-based Forest Smoke and Fire Detection model (MTL-FSFDet). Firstly, an improved Bilateral Filtering-Multi-Scale Retinex (BF-MSR) method for enhancing images was proposed, to lessen the effect of lighting on smoke images and improve the quality of the dataset. Secondly, a Hybrid Feature Extraction module, which integrates local and global information, was introduced to distinguish between targets and backgrounds, addressing smoke and fire detection in complex backgrounds. Furthermore, Dysample, a method utilizing point sampling, was designed to capture richer feature information when dealing with small targets. In addition, a feature fusion approach based on Context Gate Aggregation (CGA) was proposed to weightedly fuse low-level and high-level features, boosting the precision in detecting small targets. Finally, multi-task learning improves the capability to detect small targets and tackle complex scenarios by sharing the feature extraction module and leveraging refined supervision of the segmentation task. The findings from the experiments show that, in comparison to the baseline model, MTL-FSFDet improved the mAP@0.5 by 5.3%.