As an effective means of solving collision problems caused by the limited perspective on board, the cooperative roadside system is gaining popularity. To improve the vehicle detection abilities in such online safety systems, in this paper, we propose a novel multi-sensor multi-level enhanced convolutional network model, called multi-sensor multi-level enhanced convolutional network architecture (MME-YOLO), with consideration of hybrid realistic scene of scales, illumination, and occlusion. MME-YOLO consists of two tightly coupled structures, i.e., the enhanced inference head and the LiDAR-Image composite module. More specifically, the enhanced inference head preliminarily equips the network with stronger inference abilities for redundant visual cues by attention-guided feature selection blocks and anchor-based/anchor-free ensemble head. Furthermore, the LiDAR-Image composite module cascades the multi-level feature maps from the LiDAR subnet to the image subnet, which strengthens the generalization of the detector in complex scenarios. Compared with YOLOv3, the enhanced inference head achieves a 5.83% and 4.88% mAP improvement on visual dataset LVSH and UA-DETRAC, respectively. Integrated with the composite module, the overall architecture gains 91.63% mAP in the collected Road-side Dataset. Experiments show that even under the abnormal lightings and the inconsistent scales at evening rush hours, the proposed MME-YOLO maintains reliable recognition accuracy and robust detection performance.

In large-scale long-term dynamic environments, high-frequency dynamic objects inevitably lead to significant changes in the appearance of the scene at the same location at different times, which is catastrophic for place recognition (PR). Therefore, how to eliminate the influence of dynamic objects to achieve robust PR has universal practical value for mobile robots and autonomous vehicles. To this end, we suggest a novel semantically consistent LiDAR PR method based on chained cascade network, called SC_LPR, which mainly consists of a LiDAR semantic image inpainting network (LSI-Net) and a semantic pyramid Transformer-based PR network (SPT-Net). Specifically, LSI-Net is a coarse-to-fine generative adversarial network (GAN) with a gated convolutional autoencoder as the backbone. To effectively address the challenges posed by variable-scale dynamic object masks, we integrate the updated Transformer block with mask attention and gated trident block into LSI-Net. Sequentially, in order to generate a discriminative global descriptor representing the point cloud, we design an encoder with pyramid Transformer block to efficiently encode long-range dependencies and global contexts between different categories in the inpainted semantic image, followed by an augmented NetVALD, a generalized VLAD (Vector of Locally Aggregated Descriptors) layer that adaptively aggregates salient local features. Last but not least, we first attempt to create a LiDAR semantic inpainting dataset, called LSI-Dataset, to effectively validate the proposed method. Experimental comparisons show that our method not only improves semantic inpainting performance by about 6%, but also improves PR performance in dynamic environments by about 8% compared to the representative optimal baseline. LSI-Dataset will be publicly available at https://github.KD.LPR.com/.

In this paper, we propose a cost-effective localization solution for land vehicles, which can simultaneously adapt to the uncertain noise of inertial sensors and bridge Global Positioning System (GPS) outages. First, three Unscented Kalman filters (UKFs) with different noise covariances are introduced into the framework of Interacting Multiple Model (IMM) algorithm to form the proposed IMM-based UKF, termed as IMM-UKF. The IMM algorithm can provide a soft switching among the three UKFs and therefore adapt to different noise characteristics. Further, two IMM-UKFs are executed in parallel when GPS is available. One fuses the information of low-cost GPS, in-vehicle sensors, and micro electromechanical system (MEMS)-based reduced inertial sensor systems (RISS), while the other fuses only in-vehicle sensors and MEMS-RISS. The differences between the state vectors of the two IMM-UKFs are considered as training data of a Grey Neural Network (GNN) module, which is known for its high prediction accuracy with a limited amount of samples. The GNN module can predict and compensate position errors when GPS signals are blocked. To verify the feasibility and effectiveness of the proposed solution, road-test experiments with various driving scenarios were performed. The experimental results indicate that the proposed solution outperforms all the compared methods.

The cooperative positioning method based on wireless sensors has become a popular positioning solution for the intelligent vehicle in recent years, especially in global navigation satellite system (GNSS)-denied environments. However, the error due to the non-line-of-sight (NLOS) propagation is the main problem, which affects cooperative positioning accuracy noticeably, thereby affecting the intelligent vehicle’s functions. In order to circumvent aforementioned problem, this article proposes a multisensor cooperative fusion positioning methodology to achieve reliable position in NLOS environments. Initially, the NLOS identification module based on the consistency of multisensor measurement value is used to detect NLOS signals. Then, the NLOS mitigation module based on multilayer perceptron (MLP) is implemented to inhibit the NLOS error by decoupling the relationship between the motion of intelligent vehicle and the change of ultrawide band (UWB) observations. Lastly, the adaptive fuzzy factor graph fusion module is designed for multisensor fusion to overcome the influence of variable signal quality. The real vehicle experiments have been conducted. Compared with the traditional least squares method, the root mean square error (RMSE) is reduced from 5.932 to 1.056 m, and the maximal error (MAX) is reduced from 38.886 to 5.583 m. The results have proven the effectiveness of our proposed methodology.

Place recognition plays a crucial role in simultaneous localization and mapping. Unfortunately, however, changes in viewpoints and conditions in large-scale environments impose tricky challenges for PR. To this end, this article specifically proposes an explicit points-of-interest driven PR method, which consists of a road segmentation module based on grid-wise patch U-transformer and a PR module based on regions of interest siamese transformer NetVLAD (RI_STV). Especially for RI_STV, in the individual dimension, it is dedicated to exploring the local topological features of nonroad regions of interest. In the spatial dimension, an improved Transformer is introduced to capture the global interactions between features of interest. In the cluster dimension, NetVLAD embedded with weighted pooling is created to perform weighted aggregation of feature clusters to generate discriminative and general descriptors. Evaluation on various datasets shows that our customized method is not only impressively competitive, but also strikes the best balance between accuracy and real-time performance.

The safety and efficiency of the merging zone is particularly important for traffic networks. Although autonomous vehicle improves the safety and efficiency from vehicle view, traffic controlling in merging zone mostly focus on improving efficiency from roadside view. Lacking of detailed driving recommendation, it ignores the safety of merging zone where commercial vehicle pose a high collision risk in real traffic. This paper proposes a roadside decision-making methodology to simultaneously improve the safety and efficiency of merging zone. We have built two modules, namely assessment and decision-making. Assessment module takes advantage of Bayesian inference to evaluate dynamic collision risk. Decision-making module based on deep reinforcement learning recommends the actions to commercial vehicles by roadside unit. A series of typical simulation tests show that our method increases the TTC of commercial vehicles by an average of 62.7%. In the free flow, the overall travel time of vehicles in the merging zone is reduced by 11.68%. Most notably, when congestion occurred, the average jam length is reduced by 59.68% on the premise of safety. Moreover, the average accuracy of the roadside decision-making method on the evaluation metrics of TTC, travel time, and jam length are 93.73%, 91.65%, and 94.45%, respectively. The experimental results show that the roadside decision-making methodology simultaneously improves safety and efficiency, and it dynamically adapts free and congested traffic flow.