In automatic parking motion planning, multi-objective optimization including safety, comfort, parking efficiency, and final parking performance should be considered. Most of the current research relies on the parking data from expert drivers or prior knowledge of humans. However, it is challenging to obtain a large amount of high-quality expert drivers’ data. Furthermore, expert drivers’ data or prior knowledge of humans does not guarantee an optimal multi-objective parking performance. In this paper, we propose a model-based reinforcement learning method that learns parking policy of the data, by executing the data generation, data evaluation, and training network, iteratively. The trained network is used to guide the data generation cycle in the subsequent iteration. Based on this proposed method, we can get rid of human experience largely and learn parking strategies autonomously and quickly. The learned strategies ensure the multi-objective optimality of above requirements in the parking process. First, an environment model that approximates the actual environment is established, and the learning efficiency is accelerated through the simulated interaction between the agent and the environment model. To make the system independent of expert data or prior knowledge, a data generation algorithm combining Monte Carlo Tree Search (MCTS) and longitudinal and lateral policies is proposed. Then, to meet the multi-objective optimal demands mentioned above, a reward function is constructed to evaluate and filter the parking data. Finally, a neural network is used to learn the parking strategy from the filtered data. From the real vehicle test benchmarked with a mass-produced parking system, the proposed method is found to achieve better parking efficiency and lower requirements for start parking posture, thereby verifying the algorithm’s superiority.
Reinforcement learning (RL) is a promising direction in automated parking systems (APSs), as integrating planning and tracking control using RL can potentially maximize the overall performance. However, commonly used model-free RL requires many interactions to achieve acceptable performance, and model-based RL in APS cannot continuously learn. In this paper, a data-efficient RL method is constructed to learn from data by use of a model-based method. The proposed method uses a truncated Monte Carlo tree search to evaluate parking states and select moves. Two artificial neural networks are trained to provide the search probability of each tree branch and the final reward for each state using self-trained data. The data efficiency is enhanced by weighting exploration with parking trajectory returns, an adaptive exploration scheme, and experience augmentation with imaginary rollouts. Without human demonstrations, a novel training pipeline is also used to train the initial action guidance network and the state value network. Compared with path planning and path-following methods, the proposed integrated method can flexibly co-ordinate the longitudinal and lateral motion to park a smaller parking space in one maneuver. Its adaptability to changes in the vehicle model is verified by joint Carsim and MATLAB simulation, demonstrating that the algorithm converges within a few iterations. Finally, experiments using a real vehicle platform are used to further verify the effectiveness of the proposed method. Compared with obtaining rewards using simulation, the proposed method achieves a better final parking attitude and success rate.
Automatic parallel parking is critical to increase safety in urban narrow parking spots, maximize the traffic efficiency, and provide human drivers with mobility and convenience. Recent research integrates Monte Carlo tree search (MCTS) and artificial neural networks (ANNs) to calculate optimal lateral motions without considering the longitudinal aspect and narrow spots; advances in nonlinear programming-based (NPB) parking methods consider time-optimal parking motion in narrow spots using the time-consuming optimization calculation. To address the computational efficiency of the planning of time-optimal parking maneuvers, a complete framework relying on the two compositions was introduced. First, nonlinear optimization that formulates the minimum motion time and vehicle constraint was used to generate the data of parking motions offline. These motions were subsequently learned by ANNs. Second, the ANNs trained on the offline data were employed by an improved MCTS to generate approximate time-optimal parking motions online. The time-optimized performance and run-time performance of the proposed method were confirmed by comparing with that of NPB and other mainstream methods. A success rate of 100% for parking slots with merely 10% larger length than the vehicle was realized in simulations. Experiments were conducted on a full-sized pure electric vehicle to further confirm the effectiveness of the proposed method.
27 January 2016 in Journal of Vibration and Acoustics
Beamforming based on microphone array measurements is a popular method for identifying sound sources. However, beamforming has many limitations that limit their precision. These limitations are addressed in research. To separate the contributions which come from two sides of the microphone array more accurately, an innovative beamforming method based on a double-layer microphone array, called functional generalized inverse beamforming (FGIB), is proposed to improve beamforming performance. This method, which involves the use of a priori beamforming regularization matrix and a matrix function to redefine the inverse problem, is combined with the advantages of both generalized inverse beamforming (GIB) and functional beamforming. Compared with GIB, with reduced iterations, the computational efficiency of FGIB is greatly improved. The dynamic range of the proposed method can be modestly improved as order v increases. Furthermore, the sidelobes gradually disappear and the mainlobes become narrower. Both simulations and experiments have shown that the sources are correctly located and separated. The proposed FGIB demonstrates the good performance when compared to other beamforming methods in terms of resolution and sidelobes level.
24 February 2022 in Proceedings of the Institution of Mechanical Engineers, Part D: Journal of Automobile Engineering
Automated parking system (APS) that explicitly considers the time efficiency of the motion has received large amounts of attention in recent years. Trajectory planning module in these APS delivered parking trajectory, which was expected to be precisely tracked by tracking module. However, the reference points of frequently used trackers were selected in the spatial domain, resulting in significant trajectory tracking errors with temporal information. In this paper, a tracking control method called ILC-MPC, which combined model predictive control (MPC) and iterative learning control (ILC), was proposed to improve the spatiotemporal tracking accuracy of the autonomous vehicle. ILC was utilized for longitudinal compensation using the error signal between historical and expected speed. Accordingly, the error model in the longitudinal direction was simplified to decrease the number of decision variables in MPC. Simulation experiments using CarSim were carried out to compare the proposed method with open-loop control, linear quadratic regulator (LQR), and pure MPC that had a similar computing time with ILC-MPC. ILC-MPC converged in a few iterations of the learning process and achieved the highest tracking accuracy in spatiotemporal domain among the mentioned methods.