Convolutional neural network (CNN)-based autonomous driving object detection algorithms have excellent detection results on conventional datasets, but the detector performance can be severely degraded in low-light foggy weather environments. Existing methods have difficulty in achieving a balance between low-light image enhancement and object detection. To alleviate this problem, this paper proposes a foggy traffic environment object detection framework, IDOD-YOLOV7. This network is based on joint optimal learning of image defogging module IDOD (AOD + SAIP) and YOLOV7 detection modules. Specifically, for low-light foggy images, we propose to improve the image quality by joint optimization of image defogging (AOD) and image enhancement (SAIP), where the parameters of the SAIP module are predicted by a miniature CNN network and the AOD module performs image defogging by optimizing the atmospheric scattering model. The experimental results show that the IDOD module not only improves the image defogging quality for low-light fog images but also achieves better results in objective evaluation indexes such as PSNR and SSIM. The IDOD and YOLOV7 learn jointly in an end-to-end manner so that object detection can be performed while image enhancement is executed in a weakly supervised manner. Finally, a low-light fogged traffic image dataset (FTOD) was built by physical fogging in order to solve the domain transfer problem. The training of IDOD-YOLOV7 network by a real dataset (FTOD) improves the robustness of the model. We performed various experiments to visually and quantitatively compare our method with several state-of-the-art methods to demonstrate its superiority over the others. The IDOD-YOLOV7 algorithm not only suppresses the artifacts of low-light fog images and improves the visual effect of images but also improves the perception of autonomous driving in low-light foggy environments.

Reinforcement of motor features is necessary in action recognition tasks. In this work, we propose an efficient feature reinforcement model, termed as Feature Selection and Enhancement Networks (FEASE-Net). The core of our FEASE-Net is the use of the FEASE module to adaptively capture input features at multi-scales and reinforce them globally. FEASE module is composed of two sub-module, Feature Selection (FS) and Feature Enhancement (FE). The FS focuses on adaptive attention and selection of input features through a multi-scale structure with an attention mechanism, and FE employs channel attention to enhance the global useful feature information. To assess the effectiveness of FEASE-Net, we undertake a series of extensive experiments on two benchmark datasets, namely Kinetics 400 and Something-Something V2. Our proposed FEASE-Net can achieve a competitive performance compared with previous state-of-the-art methods that use similar backbones.

In autonomous driving, lidar has become the main vehicle sensor due to its advantages such as long-range measurement and high accuracy. However, the collected point cloud data is sparse and unevenly distributed, and it lacks characterization capabilities when facing objects with missing or similar shapes, so that the detection accuracy is low while detecting long-distance small targets with similar shapes and a small number of point clouds. In order to improve the detection accuracy of small targets represented by point clouds, this paper adopts a method that fuses point cloud and RGB image to construct a 3D object detection network architecture based on two-stage complementary fusion. In the first stage of the fusion, we use the FPS method to select some points from the raw point cloud data as key points. Then, we voxelize the raw point cloud and use the 3D sparse convolutional neural network to extract multi-scale points cloud features, which would fuse features of different scales with key points. In the second stage of fusion, a 2D object detector is used to obtain the 2D bounding box and category information of the target in the image and take the camera as the origin to extend along the direction of the 2D bounding box to form a frustum; then the point cloud and target category information within the frustum are fused into the key points. This paper uses key points as a bridge to effectively combine the semantic information of the image such as texture and color with the point cloud features. The experimental results show that the network proposed in this paper has excellent performance on small objects.

3D object detection is a crucial task for autonomous driving and robotics. However, it faces various challenges attributed to the complexity and sparsity of 3D data. LiDAR and camera are two common sensors that provide complementary information for 3D object detection. LiDAR provides accurate depth and shape information, while camera captures rich semantic and texture information. However, most existing methods for LiDAR-camera fusion either ignore the frequency domain information or use simple concatenation or element-wise operations to fuse the features, which may not fully realize the potential of both modalities. This paper proposes Frequency-aware LiDAR-Camera Fusion Networks (FAFNs) that leverage the frequency domain information to enhance the LiDAR and camera feature maps. Specifically, we design a frequency-aware feature enhancement module (FAFEM) that adaptively learns the frequency importance of each modality and assigns different weights to different frequency bands. We also design a frequency-aware multi-scale query initialization strategy that enriches the object query with frequency-wise information and multi-scale variations. We conduct the evaluation of our FAFNs using two well-established benchmarks for 3D object detection, nuScenes and Waymo, which demonstrates its competitive performance over existing state-of-the-art methods.