How to ensure robust and accurate 3D object detection under various environment is essential for autonomous driving (AD) environment perception. While, until now, most of the existing 3D object detection methods are based on the ordinary driving scenes provided by the mainstream dataset. The researches on actual complex scenes (adverse illumination, inclement weather, distant or small objects, etc.) have been ignored and there is still no comprehensive review of the recent progress in this field. Thence, this paper aims to gain a deep insight on the performance and challenges of 3D object detection methods under complex scenes for AD. Firstly, we discuss the complex driving environments in actual and the perception limitations of mainstream sensors (LIDAR and camera). Then we analyze the performance and challenges of single-modality 3D object detection methods. Therefore, in order to improve the accuracy and robustness of 3D object detection methods in some complex AD scenes, the fusion of L-C (LIDAR-camera) is recommended and systematically analyzed. Finally, some suitable datasets and potential directions are comparatively summarized to support the relative research in complex driving scenes. We hope that this review could facilitate people's research and look forward to more progress in this timely and crucial problem field.

Visual odometry (VO) is a prevalent way to deal with the relative localization problem, which is becoming increasingly mature and accurate, but it tends to be fragile under challenging environments. Comparing with classical geometry-based methods, deep learning-based methods can automatically learn effective and robust representations, such as depth, optical flow, feature, ego-motion, etc., from data without explicit computation. Nevertheless, there still lacks a thorough review of the recent advances of deep learning-based VO (Deep VO). Therefore, this paper aims to gain a deep insight on how deep learning can profit and optimize the VO systems. We first screen out a number of qualifications including accuracy, efficiency, scalability, dynamicity, practicability, and extensibility, and employ them as the criteria. Then, using the offered criteria as the uniform measurements, we detailedly evaluate and discuss how deep learning improves the performance of VO from the aspects of depth estimation, feature extraction and matching, pose estimation. We also summarize the complicated and emerging areas of Deep VO, such as mobile robots, medical robots, augmented and virtual reality, etc. Through the literature decomposition, analysis, and comparison, we finally put forward a number of open issues and raise some future research directions in this field.

Video anomaly detection aims to detect the segments containing abnormal events from video sequence, which is a current research hotspot due to the importance in maintaining social security. Recent detection methods tend to build frame reconstruction or frame prediction model based on deep learning to learn features of events. The reconstruction-based methods reproduce the input frame one-to-one, inevitably losing some temporal features. The prediction-based methods predict frames according to the natural time order, but ignore the reverse time information, causing the deviation in information learning. Besides, anomaly evaluation methods based on patch-level error neglect the diversity of object sizes in complex scenes, and it is difficult to determine the optimal size of the error patch accurately. For these issues, we propose a bidirectional spatio-temporal feature learning framework with multi-scale anomaly evaluation strategy. A video sequence is input to a double-encoder double-decoder network, and bidirectional spatio-temporal features are obtained for bidirectional prediction by fusing forward and backward features extracted from the two encoders. The multi-scale anomaly evaluation method is implemented based on error pyramid and mean pooling, which effectively detects target objects with different sizes. Experiments on several publicly video datasets show that our method outperforms most of existing methods.

Most visual simultaneous localization and mapping (SLAM) algorithms rely on geometric features, such as points, lines, and planes, and these algorithms treated all features as equally important in the optimization process, ignoring the semantic information of these features. However, the robust and rich semantic features should play a more important role, which is similar to the mechanism of visual saliency or visual attention. Therefore, this article aims to mimic this mechanism in the visual SLAM framework by using the saliency prediction model. To solve the center bias of the generic salient dataset, we proposed a method for recomputing saliency map by considering both geometric and semantic information. Then, we proposed the salient bundle adjustment (SBA) algorithm by using the value of saliency prediction map as the weight of the feature points in the traditional bundle adjustment (BA) approach. Finally, exhaustive experiments conducted with the state-of-the-art algorithm show that our proposed algorithm outperforms existing algorithms such as direct sparse odometry (DSO) and oriented brief (ORB)-SLAM3 in both indoor and outdoor environments.

In the study of image compressed sensing (CS), various priors are explored for regularization to achieve better reconstruction and provide different additional information. However, it is a challenge to combine different priors for better performance. In this article, we propose a novel framework (dubbed SCS-GNet) that integrates the merits of multiple priors for image CS. The framework is composed of a measurement matrix generator network and a reconstruction generator network, in which the implicit image prior is explored in the process of measurement matrix generation, and deep image and sparsity priors are explored in the reconstruction process. The two generator networks are separate in SCS-GNet, and the partial consistency of loss functions is explored for connection to boost the performance of image CS. Different from existing end-to-end joint training deep network-based methods, our measurement matrix generation process is performed separately, while the reconstruction process is training-free. Moreover, a novel training strategy is applied in the process of measurement matrix generation to enable the framework to be available for different measurement rates. Extensive experiments demonstrate that the proposed SCS-GNet achieves adaptive and scalable measurement and reconstruction at different measurement rates and offers competitive performance compared with other state-of-the-art methods.