One of the most challenging problems in human-machine co-work is the gap between human intention and the machine's understanding and execution. Large Language Models (LLMs) have been showing superior abilities in solving such issue. In this paper, we design a universal framework that embeds LLMs as a vehicle “Co-Pilot” of driving, which can accomplish specific driving tasks with human intention satisfied based on the information provided. Meanwhile, a utilization workflow is defined to handle the interaction between humans and vehicles, and memory mechanism is introduced to organize the information involved in the tasks. Expert Oriented Black-Box tuning is proposed to improve the performance of the Co-Pilot without finetuning or training the LLMs. In the experiment, the Co-Pilot is applied to two different tasks, i.e., path tracking control and trajectory planning. The Co-Pilot adjusts vehicle operating conditions by selecting a proper controller or planning a certain trajectory to fit human intentions. Simulation tests are conducted to evaluate the performance and generality of the proposed module. The results show that the Co-Pilot can accomplish most of the tasks based on only natural language processing, although it is not flawless. Finally, a discussion about human-machine hybrid intelligence and further applications of LLMs in autonomous driving is made. We believe that such a framework has promising potential in further applications in the field of automous vehicles.

Accurate calibration between LiDAR and camera sensors is crucial for autonomous driving systems to perceive and understand the environment effectively. Typically, LiDAR–camera extrinsic calibration requires feature alignment and overlapping fields of view. Aligning features from different modalities can be challenging due to noise influence. Therefore, this paper proposes a targetless extrinsic calibration method for monocular cameras and LiDAR sensors that have a non-overlapping field of view. The proposed solution uses pose transformation to establish data association across different modalities. This conversion turns the calibration problem into an optimization problem within a visual SLAM system without requiring overlapping views. To improve performance, line features serve as constraints in visual SLAM. Accurate positions of line segments are obtained by utilizing an extended photometric error optimization method. Moreover, a strategy is proposed for selecting appropriate calibration methods from among several alternative optimization schemes. This adaptive calibration method selection strategy ensures robust calibration performance in urban autonomous driving scenarios with varying lighting and environmental textures while avoiding failures and excessive bias that may result from relying on a single approach.

Extreme operating conditions refer to the critical dynamic state during vehicle operation. The lack of experimental data under critical conditions is one of the fundamental problems in the study. To solve the problem, we design an LSTM-VAE based generating model to generate rational control sequences that can push vehicles toward extreme operating conditions and used simulation tests to analyze them. Specifically, we train the Encoder to study the basic driving logic of the control sequences collected during free-drive tests by human drivers, forming a low-dimension latent feature space. Then, we sample from specified regions in the latent feature space and use the Decoder to generate new control sequences. Finally, we use the sequences as the control input of the 27-DoF high-precision vehicle dynamic simulation platform and analyze the variations of simulated vehicle dynamics. We conduct different experiments and validate the method from different aspects. Results reveal that by sampling from specific regions of the latent feature space, we get a higher chance to generate desired control sequences for extreme operating conditions.

This paper proposes an energy-saving lane-changing trajectory model for intelligent electric vehicles at high speeds that considers different driving needs under multiple constraints to address the issues of simple lane-changing considerations and poor safety. An economic index is added to construct a multi-objective optimization function based on the comfort, safety, and efficiency of lane changing. The particle swarm optimization algorithm is used to solve the optimal lane-changing time. The weight of the index is obtained by analyzing different driving needs using the Analytic Hierarchy Process in different scenarios. The multi-objective function is adjusted to plan the optimal lane-changing trajectory that meets driving needs. The simulation shows that the proposed model can generate smooth and feasible lane-changing trajectories that meet different driving needs. The energy consumption analysis results indicate that the construction of economic indicators can effectively reduce the energy consumption of vehicles driving on highways. The tracking analysis results indicate that the target vehicle can smoothly and safely change lanes on the planned trajectory, verifying the effectiveness and rationality of the planned trajectory.