The visible image contains many high-frequency components that provide texture details with high spatial resolution and definition consistent with human visual perception, but it is easily affected by external factors such as light, weather, and obstructions. On the other hand, the infrared image is a radiation image whose contrast is determined by the temperature difference between the target and the background, and is not easily affected by external conditions. Integrating complementary information from both image types into one image is therefore very useful. In our paper, we propose a dual-frequency cross-enhanced fusion network called DCFusion for infrared and visible image fusion. We design a frequency decomposition module and a frequency enhancement module based on Laplacian of Gaussian for feature decomposition and enhancement, respectively. We then build a dual-frequency cross-enhanced fusion generator network based on these two modules to achieve enhanced fusion. We also use the sum of visible and infrared discriminator and the visible discriminator to balance our fusion results, replacing the traditional single visible discriminator. Our method is an end-to-end model, avoiding the manual design of complex fusion rules like traditional methods. Compared with existing advanced fusion algorithms, our method outperforms most of them in qualitative comparison, quantitative comparison, and target detection accuracy. Finally, the experiment proves that our method can effectively enhance the fusion of the target scene even in harsh environments such as complex lighting, low illumination, and smoke scenes.

The infrared and visible cross-modal registration and fusion can generate more comprehensive representations of object and scene information. Previous frameworks primarily focus on addressing the modality disparities and the impact of preserving diverse modality information on the performance of registration and fusion tasks among different static image pairs. However, these frameworks overlook the practical deployment on real-world devices, particularly in the context of video streams. Consequently, the resulting video streams often suffer from instability in registration and fusion, characterized by fusion artifacts and inter-frame jitter. In light of these considerations, this paper proposes a unified registration and fusion scheme for video streams, termed RCVS. It utilizes a robust matcher and spatial-temporal calibration module to achieve stable registration of video sequences. Subsequently, RCVS combines a fast lightweight fusion network to provide stable fusion video streams for infrared and visible imaging. Additionally, we collect a infrared and visible video dataset HDO, which comprises high-quality infrared and visible video data captured across diverse scenes. Our RCVS exhibits superior performance in video stream registration and fusion tasks, adapting well to real-world demands. Overall, our proposed framework and HDO dataset offer the first effective and comprehensive benchmark in this field, solving stability and real-time challenges in infrared and visible video stream fusion while assessing different solution performances to foster development in this area. The HDO dataset is publicly available on https://github.com/xiehousheng/HDO .

Ensuring compliance with safety regulations regarding wearing is essential for the safety and security of those working on substation construction sites. However, relying on supervisors to monitor workers in real time on the work site or through remote surveillance videos is both unreasonable and inefficient. A deep learning network approach named FFA-YOLOv7 is presented in this study that utilizes an improved version of YOLOv7 to detect violations of worker wearing in real time during power construction site surveillance. In YOLOv7, the feature pyramid network (FPN) of the neck stage is constructed through continuous upsampling and skip connections for feature fusion, after continuous downsampling of the backbone. However, this process can result in the loss of precise shallow position information. To tackle this issue, we have introduced a novel feature fusion pathway to the FPN architecture, enabling each layer not only to fuse feature maps from the same level during the downsampling course but also to fuse feature maps from shallower levels. This approach combines precise positional information from shallow layers with rich semantic information from deep layers. Additionally, we utilized attention after feature fusion in each layer to optimize the feature map fusion effect and achieve better detection accuracy performance. In order to conduct comparative experiments, we trained six variations of the YOLO model as detectors using a dataset gathered from realistic construction sites. The experimental results indicate that our proposed FFA-YOLOv7 attained a detection precision of 95.92% and a recall rate of 97.13%, demonstrating a high level of accuracy and a low rate of missed detections. These outcomes effectively satisfy the requirements for robust and accurate detection of real-world power construction violations.