Forest fires have continually endangered personal safety and social property. To reduce the occurrences of forest fires, it is essential to detect forest fire smoke accurately and quickly. Traditional forest fire smoke detection based on convolutional neural networks (CNNs) needs many hand-designed components and shows poor ability to detect small and inconspicuous smoke in complex forest scenes. Therefore, we propose an improved early forest fire smoke detection model based on deformable transformer for end-to-end object detection (deformable DETR). We use deformable DETR as a baseline containing the best sparse spatial sampling for smoke with deformable convolution and relation modeling capability of the transformer. We integrate a Multi-scale Context Contrasted Local Feature module (MCCL) and a Dense Pyramid Pooling module (DPPM) into the feature extraction module for perceiving features of small or inconspicuous smoke. To improve detection accuracy and reduce false and missed detections, we propose an iterative bounding box combination method to generate precise bounding boxes which can cover the entire smoke object. In addition, we evaluate the proposed approach using a quantitative and qualitative self-made forest fire smoke dataset, which includes forest fire smoke images of different scales. Extensive experiments show that our improved model’s forest fire smoke detection accuracy is significantly higher than that of the mainstream models. Compared with deformable DETR, our model shows better performance with improvement of mAP (mean average precision) by 4.2%, AP_S (AP for small objects) by 5.1%, and other metrics by 2% to 3%. Our model is adequate for early forest fire smoke detection with high detection accuracy of different-scale smoke objects.

Deep learning-based forest fire vision monitoring methods have developed rapidly and are becoming mainstream. The existing methods, however, are based on enormous amounts of data, and have issues with weak feature extraction, poor small target recognition and many missed and false detections in complex forest scenes. In order to solve these problems, we proposed a multi-task learning-based forest fire detection model (MTL-FFDet), which contains three tasks (the detection task, the segmentation task and the classification task) and shares the feature extraction module. In addition, to improve detection accuracy and decrease missed and false detections, we proposed the joint multi-task non-maximum suppression (NMS) processing algorithm that fully utilizes the advantages of each task. Furthermore, considering the objective fact that divided flame targets in an image are still flame targets, our proposed data augmentation strategy of a diagonal swap of random origin is a good remedy for the poor detection effect caused by small fire targets. Experiments showed that our model outperforms YOLOv5-s in terms of $mAP$ (mean average precision) by 3.2%, $A{P}_S$ (average precision for small objects) by 4.8%, $A{R}_S$ (average recall for small objects) by 4.0%, and other metrics by 1% to 2%. Finally, the visualization analysis showed that our multi-task model can focus on the target region better than the single-task model during feature extraction, with superior extraction ability.

Forest resources are one of the indispensable resources of the earth, which are the basis for the survival and development of human society. With the swift advancements in computer vision and artificial intelligence technology, the utilization of deep learning for smoke detection has achieved remarkable results. However, the existing deep learning models have poor performance in forest scenes and are difficult to deploy because of numerous parameters. Hence, we introduce an optimized forest fire smoke monitoring system for embedded edge devices based on a lightweight deep learning model. The model makes full use of the multi-scale variable attention mechanism of Transformer architecture to strengthen the ability of image feature extraction. Considering the needs of application scenarios, we propose an improved lightweight network model LCNet for feature extraction, which can reduce the parameters and enhance detecting ability. In order to improve running speed, a simple semi-supervised label knowledge distillation scheme is used to enhance the overall detection capability. Finally, we design and implement a forest fire smoke detection system on an embedded device, including the Jetson NX hardware platform, high-definition camera, and detection software system. The lightweight model is transplanted to the embedded edge device to achieve rapid forest fire smoke detection. Also, an asynchronous processing framework is designed to make the system highly available and robust. The improved model reduces three-fourths of the parameters and increases speed by 3.4 times with similar accuracy to the original model. This demonstrates that our system meets the precision demand and detects smoke in time.