On-ramp merging is a challenging task for autonomous vehicles (AVs), especially in mixed traffic where AVs coexist with human-driven vehicles (HDVs). In this paper, we formulate the mixed-traffic highway on-ramp merging problem as a multi-agent reinforcement learning (MARL) problem, where the AVs (on both merge lane and through lane) collaboratively learn a policy to adapt to HDVs to maximize the traffic throughput. We develop an efficient and scalable MARL framework that can be used in dynamic traffic where the communication topology could be time-varying. Parameter sharing and local rewards are exploited to foster inter-agent cooperation while achieving great scalability. An action masking scheme is employed to improve learning efficiency by filtering out invalid/unsafe actions at each step. In addition, a novel priority-based safety supervisor is developed to significantly reduce collision rate and greatly expedite the training process. A gym-like simulation environment is developed and open-sourced with three different levels of traffic densities. We exploit curriculum learning to efficiently learn harder tasks from trained models under simpler settings. Comprehensive experimental results show the proposed MARL framework consistently outperforms several state-of-the-art benchmarks.
15 May 2023 in IEEE Transactions on Intelligent Vehicles
One of the major obstacles along the way of electric vehicles' (EVs') wider global adoption is their limited driving range. Extreme cold or hot environments can further impact the EV's range as a significant amount of energy is needed for cabin and battery temperature regulation while the battery's power and energy capacity are also impeded. To overcome this issue, we present an optimal control strategy based on nonlinear model predictive control (NMPC) for integrated thermal management (ITM) of the battery and cabin of EVs, where the proposed NMPC simultaneously optimizes the EV range and cabin comfort in real time. Firstly, the components of the designed ITM system are introduced and control-oriented modeling is done. Secondly, to demonstrate and validate the benefits of the proposed ITM, an optimal control problem is defined and dynamic programming (DP) is employed to find the global optimal solution. Thirdly, for practical implementation, NMPC-based control strategy is developed, where the cost function design and weights calibration are done in comparison with DP global optimal solution. Weight-tuning results show that our NMPC-based approach can achieve close driving range maximization as compared to the DP benchmark while ensuring cabin comfort. The developed NMPC-based ITM strategy is further illustrated by comparing its performance to two additional benchmark strategies, i.e., rule-based control and cabin heating only. Finally, our simulation results also identify several important factors that impact the benefits of the proposed NMPC-based ITM, which are used to summarize the operating conditions under which the proposed ITM is critically needed.
16 March 2022 in IEEE Transactions on Vehicular Technology
Simultaneous vibration control and energy harvesting of vehicle suspensions have attracted significant research attention over the past decades. However, existing energy harvesting shock absorbers (EHSAs) compromise suspension performance for high-efficiency energy harvesting and are only responsive to narrow-bandwidth vibrations. In this paper, we propose a new ball-screw-based EHSA design -- inerter pendulum vibration absorber (IPVA) -- that integrates an electromagnetic rotary EHSA with a nonlinear pendulum vibration absorber. We show that this design simultaneously improves ride comfort and energy harvesting efficiency by exploiting the nonlinear effects of pendulum inertia. To further improve the performance, we develop a novel stochastic linearization model predictive control (SL-MPC) approach in which we employ stochastic linearization to approximate the nonlinear dynamics of EHSA that has superior accuracy compared to standard linearization. In particular, we develop a new stochastic linearization method with guaranteed stabilizability, which is a prerequisite for control designs. This leads to an MPC problem that is much more computationally efficient than the nonlinear MPC counterpart with no major performance degradation. Also, the effect of different road preview configurations on control performance is investigated, which is shown to have a significant impact on control performance. Extensive simulations are performed to show the superiority of the proposed new nonlinear EHSA and to demonstrate the efficacy of the proposed SL-MPC.
27 January 2022 in IEEE Transactions on Intelligent Vehicles
In this paper, we develop a mechanical system inspired microscopic traffic model to characterize the longitudinal interaction dynamics among a chain of vehicles. In particular, we extend our prior work on mass-spring-damper-clutch based car-following model between two vehicles to multivehicle scenario. This model can naturally capture the drivers tendency to maintain the same speed as the vehicle ahead while keeping a (speed-dependent) desired spacing. It is also capable of characterizing the impact of the following vehicle on the preceding vehicle, which is generally neglected in existing models. A new string stability criterion is defined for the considered multi-vehicle dynamics, and stability analysis is performed on the system parameters and time delays. An efficient online parameter identification algorithm, sequential recursive least squares with inverse QR decomposition (SRLS-IQR), is developed to estimate the driving-related model parameters. These real-time estimated parameters can be employed in advanced longitudinal control systems to enable accurate prediction of vehicle trajectories for improved safety and fuel efficiency. The proposed model and the parameter identification algorithm are validated on NGSIM, a naturalistic driving dataset, as well as our own connected vehicle driving data. Promising performance is demonstrated.
1 August 2022 in IEEE Transactions on Intelligent Transportation Systems
There is an increasing popularity in exploiting modern vehicles as mobile sensors to obtain important road information such as potholes, black ice and road profile. Availability of such information has been identified as a key enabler for next-generation vehicles with enhanced safety, efficiency, and comfort. However, existing road information discovery approaches have been predominately performed in a single-vehicle setting, which is inevitably susceptible to vehicle model uncertainty and measurement errors. To overcome these limitations, this paper presents a novel cloud-assisted collaborative estimation framework that can utilize multiple heterogeneous vehicles to iteratively enhance estimation performance. Specifically, each vehicle combines its onboard measurements with a cloud-based Gaussian process (GP), crowdsourced from prior participating vehicles as “pseudo-measurements”, into a local estimator to refine the estimation. The resultant local onboard estimation is then sent back to the cloud to update the GP, where we utilize a noisy input GP (NIGP) method to explicitly handle uncertain GPS measurements. We employ the proposed framework to the application of collaborative road profile estimation. Promising results on extensive simulations and hardware-in-the-loop experiments show that the proposed collaborative estimation can significantly enhance estimation and iteratively improve the performance from vehicle to vehicle, despite vehicle heterogeneity, model uncertainty, and measurement noises.
3 August 2022 in Journal of Dynamic Systems, Measurement, and Control
Road profile information can be utilized to enhance vehicle control performance, passenger ride comfort, and route planning and optimization. Existing road-profile estimation algorithms are mainly based on one single vehicle, which are usually susceptible to modeling uncertainties and measurement noises. This technical brief proposes a new cascaded learning framework that utilizes multiple heterogeneous vehicles to achieve enhanced estimation. In this framework, each individual vehicle first performs a local estimation via a standard disturbance observer (DOB) while traversing a considered road segment. Then learning filters are designed to dynamically connect the vehicles, and the preliminary estimates from one vehicle are utilized to generate the learning signal for another. For each vehicle, a heterogeneous learning signal is produced and added to its estimation loop for estimating enhancement, through which the estimations are improved over multiple iterations. Extensive numerical studies are carried out to validate the effectiveness of the proposed method with promising results demonstrated.