20 June 2017 in IEEE Transactions on Cognitive and Developmental Systems
The perception-driven approach and end-to-end system are two major vision-based frameworks for self-driving cars. However, it is difficult to introduce attention and historical information into the autonomous driving process, which are essential for achieving human-like driving in these two methods. In this paper, we propose a novel model for self-driving cars called the brain-inspired cognitive model with attention. This model comprises three parts: 1) a convolutional neural network for simulating the human visual cortex; 2) a cognitive map to describe the relationships between objects in a complex traffic scene; and 3) a recurrent neural network, which is combined with the real-time updated cognitive map to implement the attention mechanism and long-short term memory. An advantage of our model is that it can accurately solve three tasks simultaneously: 1) detecting the free space and boundaries for the current and adjacent lanes; 2) estimating the distances to obstacles and vehicle attitude; and 3) learning the driving behavior and decision-making process of a human driver. Importantly, the proposed model can accept external navigation instructions during an end-to-end driving process. To evaluate the model, we built a large-scale road-vehicle dataset containing over 40000 labeled road images captured by three cameras placed on our self-driving car. Moreover, human driving activities and vehicle states were recorded at the same time.
28 May 2019 in IEEE Transactions on Intelligent Vehicles
Effective testing and simulation environment is a vital part in the research of self-driving cars, which can test self-driving software and hardware quickly in different virtual environments at low cost. However, as for the mainstream simulation platforms, there is a certain gap between the virtual environment and the actual self-driving platform, which decreases the efficiency of development and makes it difficult to complete the migration from the virtual scenario to the real environment. Therefore, in this paper, we proposed a novel hardware-in-the-loop (HiL) simulation system. It takes the real hardware ECU (Electronic Control Unit) of self-driving vehicle as a part of the simulation platform, which improves the efficiency of development and testing, and also, the tested algorithm can be implemented into self-driving cars faster. The proposed HiL simulation system mainly consists of four parts: (1) the vehicle kinematic model simulation (2) the multi-sensor simulation (3) the environment simulation and (4) the ECU hardware. Our system is able to perform the closed-loop algorithm verification such as perception, planning, decision-making and control in virtual scenes. Furthermore, the algorithms tested in the simulation environment can be deployed rapidly to the real self-driving vehicles.
21 April 2021 in IEEE Robotics and Automation Letters
The path planning of mobile robots is an optimization problem that is difficult to solve directly owing to its nonlinear characteristics. This letter proposes the “global-local” Coupling Two-Stage Path Planning (CTSP) method. First, the globally optimal solution in the configuration space is given by the global planner. Then, in the local planning stage, the optimal solution of the local environment is constantly searched, guided by the prior information of the globally optimal solution. The strategy used in the global planning stage is the iterative optimization method based on an initial solution. The local planning stage adopts the sampling-evaluation strategy, that is, sampling the candidate paths and then using the evaluation function to perform path selection. The proposed method has two innovations: 1) a novel global iterative optimization method is proposed and 2) a new cost function for evaluating the sampled paths is constructed, which improves the coupling of the global and local paths. We implement and test this method in a simulation environment, where the experimental results verify the effectiveness of the proposed method.
24 December 2021 in IEEE Transactions on Intelligent Transportation Systems
The core value of simulation-based autonomy tests is to create densely extreme traffic scenarios to test the performance and robustness of the algorithms and systems. Test scenarios are usually designed or extracted manually from the real-world data, which is inefficient with a remarkable domain gap compared with testing in real scenarios. Therefore, it is crucial to automatically generate realistic and diverse dynamic traffic scenarios making autonomy tests efficient. Moreover, scenario generation is expected to be interpretable, controllable, and diversified, which can be hard to achieve simultaneously by methods based on rules or deep networks. In this paper, we propose a dynamic traffic scenario generation method called SceGene, inspired by genetic inheritance and mutation processes in biological intelligence. SceGene applies biological processes, such as crossover and mutation, to exchange and mutate the content of scenarios, and involves the natural selection process to control generation direction. SceGene has three main parts: 1) a new representation method for describing the traffic scenarios' feature; 2) a new scenario generation algorithm based on crossover, mutation, and selection; and 3) an abnormal scenario information repair method based on the microscopic driving model. Evaluation on the public traffic scenario dataset shows that SceGene can ensure highly realistic and diversified scenario generation in an interpretable and controllable way, significantly improving the efficiency of the simulation-based autonomy tests.
16 December 2020 in IEEE Transactions on Intelligent Transportation Systems
Autonomous driving systems (ADSs) need to be able to respond quickly to changes in the dynamic traffic scenario. However, regardless of the changes occurring in traffic scenes, the current local path planning frameworks of ADSs are based on the fixed frequency re-planning path (i.e., running their planning algorithms repeatedly). This planning method makes it difficult to provide a reasonable traveling path, agility, and comfort for driverless vehicles in changing traffic scenarios. Therefore, this article performs an in-depth analysis of the problems of traditional planning frameworks which use a fixed frequency to replan the path and proposes a novel path planning framework that is universal based on multiple-models. The proposed framework divides the planning process into several layers, each of which has different functions. With this framework, the ADS can adaptively adjust the planning process according to the changes in traffic scenes and then provide different path planning algorithms to ensure its safety and flexibility in the process of driving. Moreover, the problems caused by the traditional planning framework can be solved. This framework has been applied to the autonomous vehicle ``Pioneer'', which won first place in the 2019 China Intelligent Vehicle Future Challenge (IVFC). The effectiveness and rationality of the integrated framework of local path planning proposed in this article were verified by a large number of tests in real-world traffic scenarios.
22 March 2022 in IEEE Transactions on Cybernetics
Localization is a fundamental and crucial module for autonomous vehicles. Most of the existing localization methodologies, such as signal-dependent methods (RTK-GPS and Bluetooth), simultaneous localization and mapping (SLAM), and map-based methods, have been utilized in outdoor autonomous driving vehicles and indoor robot positioning. However, they suffer from severe limitations, such as signal-blocked scenes of GPS, computing resource occupation explosion in large-scale scenarios, intolerable time delay, and registration divergence of SLAM/map-based methods. In this article, a self-localization framework, without relying on GPS or any other wireless signals, is proposed. We demonstrate that the proposed homogeneous normal distribution transform algorithm and two-way information interaction mechanism could achieve centimeter-level localization accuracy, which reaches the requirement of autonomous vehicle localization for instantaneity and robustness. In addition, benefitting from hardware and software co-design, the proposed localization approach is extremely light-weighted enough to be operated on an embedded computing system, which is different from other LiDAR localization methods relying on high-performance CPU/GPU. Experiments on a public dataset (Baidu Apollo SouthBay dataset) and real-world verified the effectiveness and advantages of our approach compared with other similar algorithms.