Highly integrated information sharing among people, vehicles, roads, and cloud systems, along with the rapid development of autonomous driving technologies, has spurred the evolution of automobiles from simple “transportation tools” to interconnected “intelligent systems”. The intelligent cockpit is a comprehensive application space for various new technologies in intelligent vehicles, encompassing the domains of driving control, riding comfort, and infotainment. It provides drivers and passengers with safety, comfort, and pleasant driving experiences, serving as the gateway for traditional automobile manufacturing to upgrade towards an intelligent automotive industry ecosystem. This is the optimal convergence point for the intelligence, connectivity, electrification, and sharing of automobiles. Currently, the form, functions, and interaction methods of the intelligent cockpit are gradually changing, transitioning from the traditional “human adapts to the vehicle” viewpoint to the “vehicle adapts to human”, and evolving towards a future of natural interactive services where “humans and vehicles mutually adapt”. This article reviews the definitions, intelligence levels, functional domains, and technical frameworks of intelligent automotive cockpits. Additionally, combining the core mechanisms of human–machine interactions in intelligent cockpits, this article proposes an intelligent-cockpit human–machine interaction process and summarizes the current state of key technologies in intelligent-cockpit human–machine interactions. Lastly, this article analyzes the current challenges faced in the field of intelligent cockpits and forecasts future trends in intelligent cockpit technologies.

Conditionally automated vehicles can be operated on most regular roads without a driver’s supervision. They show excellent potential for market adoption and are now being targeted by numerous auto manufacturers for mass production. The system of such a vehicle enables it to autonomously perform dynamic driving tasks within the operational design domain, but once this system fails or malfunctions, the vehicle will be unable to reliably complete a dynamic driving task. In such cases, the system will send a takeover request, following which the driver needs to immediately take control of the vehicle. The driver’s physical and mental state, as well as the non-driving-related tasks that they are engaged in, affects the time required for them to perform the takeover and the quality of the takeover. To manage driving risks and guarantee the safety of drivers during automated driving, an automated driving system should be able to monitor a driver’s state and behavior, assess their level of alertness, and perform the appropriate actions as required. In recent years, techniques for monitoring a driver’s state have been widely researched, and several practical methods have been proposed. In this review, we review representative methods, aiming to introduce the concept of driver state monitoring to a broader audience. First, we identified a few typical driver states that are important in driver state monitoring from the perspective of application demands for driver state monitoring in conditionally automated driving. Then, we categorized and reviewed existing studies on driver state monitoring according to the types of sensing data employed by previously proposed methods. Additionally, we collected datasets corresponding to different data types for driver state monitoring. Finally, by analyzing existing issues in driver state monitoring in relation to conditionally automated driving, we provided several suggestions for future research directions in this area, and discussed potential challenges and possible solutions.

The cockpit is evolving from passive, reactive interaction toward proactive, cognitive interaction, making precise predictions of driver intent a key factor in enhancing proactive interaction experiences. This paper introduces Cockpit-Llama, a novel language model specifically designed for predicting driver behavior intent. Cockpit-Llama predicts driver intent based on the relationship between current driver actions, historical interactions, and the states of the driver and cockpit environment, thereby supporting further proactive interaction decisions. To improve the accuracy and rationality of Cockpit-Llama’s predictions, we construct a new multi-attribute cockpit dataset that includes extensive historical interactions and multi-attribute states, such as driver emotional states, driving activity scenarios, vehicle motion states, body states and external environment, to support the fine-tuning of Cockpit-Llama. During fine-tuning, we adopt the Low-Rank Adaptation (LoRA) method to efficiently optimize the parameters of the Llama3-8b-Instruct model, significantly reducing training costs. Extensive experiments on the multi-attribute cockpit dataset demonstrate that Cockpit-Llama’s prediction performance surpasses other advanced methods, achieving BLEU-4, ROUGE-1, ROUGE-2, and ROUGE-L scores of 71.32, 80.01, 76.89, and 81.42, respectively, with relative improvements of 92.34%, 183.61%, 95.54%, and 201.27% compared to ChatGPT-4. This significantly enhances the reasoning and interpretative capabilities of intelligent cockpits.