In this paper, we present a new effective infrared (IR) and visible (VIS) image fusion method by using a deep neural network. In our method, a Siamese convolutional neural network (CNN) is applied to automatically generate a weight map which represents the saliency of each pixel for a pair of source images. A CNN plays a role in automatic encoding an image into a feature domain for classification. By applying the proposed method, the key problems in image fusion, which are the activity level measurement and fusion rule design, can be figured out in one shot. The fusion is carried out through the multi-scale image decomposition based on wavelet transform, and the reconstruction result is more perceptual to a human visual system. In addition, the visual qualitative effectiveness of the proposed fusion method is evaluated by comparing pedestrian detection results with other methods, by using the YOLOv3 object detector using a public benchmark dataset. The experimental results show that our proposed method showed competitive results in terms of both quantitative assessment and visual quality.

In this study, a novel multi‐layer fused convolution neural network (MLF‐CNN) is proposed for detecting pedestrians under adverse illumination conditions. Currently, most existing pedestrian detectors are very likely to be stuck under adverse illumination circumstances such as shadows, overexposure, or nighttime. To detect pedestrians under such conditions, the authors apply deep learning for effective fusion of the visible and thermal information in multispectral images. The MLF‐CNN consists of a proposal generation stage and a detection stage. In the first stage, they design an MLF region proposal network and propose to use summation fusion method for integration of the two convolutional layers. This combination can detect pedestrians in different scales, even in adverse illumination. Furthermore, instead of extracting features from a single layer, they extract features from three feature maps and match the scale using the fused ROI pooling layers. This new multiple‐layer fusion technique can significantly reduce the detection miss rate. Extensive evaluations of several challenging datasets well demonstrate that their approach achieves state‐of‐the‐art performance. For example, their method performs 28.62% better than the baseline method and 11.35% better than the well‐known faster R‐CNN halfway fusion method in detection accuracy on KAIST multispectral pedestrian dataset.

Pedestrian-related accidents are much more likely to occur during nighttime when visible (VI) cameras are much less effective. Unlike VI cameras, infrared (IR) cameras can work in total darkness. However, IR images have several drawbacks, such as low-resolution, noise, and thermal energy characteristics that can differ depending on the weather. To overcome these drawbacks, we propose an IR camera system to identify pedestrians at night that uses a novel attention-guided encoder-decoder convolutional neural network (AED-CNN). In AED-CNN, encoder-decoder modules are introduced to generate multi-scale features, in which new skip connection blocks are incorporated into the decoder to combine the feature maps from the encoder and decoder module. This new architecture increases context information which is helpful for extracting discriminative features from low-resolution and noisy IR images. Furthermore, we propose an attention module to re-weight the multi-scale features generated by the encoder-decoder module. The attention mechanism effectively highlights pedestrians while eliminating background interference, which helps to detect pedestrians under various weather conditions. Empirical experiments on two challenging datasets fully demonstrate that our method shows superior performance. Our approach significantly improves the precision of the state-of-the-art method by 5.1% and 23.78% on the Keimyung University (KMU) and Computer Vision Center (CVC)-09 pedestrian dataset, respectively.

The authors propose a novel and efficient method for single image dehazing. To accelerate the transmission estimation process, a block‐to‐pixel interpolation method is used for fine dark channel computation, in which the block‐level dark channel is first computed, and then the fine pixel‐level dark channel is obtained by a weighted voting of the block‐level dark channel to preserve edges and smooth out texture noise. This technique can be used for a direct transmission map generation without a computationally expensive refinement step. Since the dark channel prior (DCP) is not valid in bright (sky) regions, they propose an adaptive DCP modelled by a Gaussian curve that produces a more natural recovered image of the sky and other bright regions. In addition, a scaling method for transmission map computation is proposed to further accelerate the dehazing method. Through experiments, they show that the proposed adaptive block‐to‐pixel technique is about 30 times faster and produces improved recovered images than the well‐known state‐of‐the‐art DCP approach.

The vision‐based lane detection is an important component of advanced driver assistance systems and it is essential for lane departure warning, lane keeping, and vehicle localisation. However, it is a challenging problem to improve the robustness of multi‐lane detection due to factors, such as perspective effect, possible low visibility of lanes, and partial occlusions. To deal with these issues, the authors propose an improved lane hypothesis generation method using a reliable binary blob analysis. Most existing top‐view based methods focused on the lane model fitting, but they neglected the reliability of hypothesis generation and the effectiveness in challenging conditions. To cope with these shortcomings, the authors carried out vanishing point detection and inverse perspective mapping to remove the perspective effect from the road images. Then two‐stage binary blob filtering and blob verification techniques using classification are introduced to improve the robustness of lane hypothesis generation for lane detection. The experimental results show that the average detection accuracy on a new challenging multi‐lane dataset is 97.7%. The performance of the proposed method outperforms that of the state‐of‐the‐art method by 1.6% in detection accuracy on the Caltech lane benchmark dataset.