Cities of all sizes around the world are facing increasing levels of congestion, which leads to increasing travel time and emissions, and ultimately affects the quality of life. Relevant research suggests that adaptive traffic light control systems can improve the traffic flow, but their impact on energy-efficiency of vehicle propulsion systems is not well understood. In this study, we use Proximal Policy Optimization, a Deep Reinforcement Learning algorithm, to develop an optimized adaptive traffic light control systems that controls three traffic lights simultaneously. For this purpose, we have created a microscopic traffic simulation of the city of Aachen, Germany, calibrated on the basis of traffic measurements, where the actual traffic light schedule, a green-wave, fixed-time control scheme, serves as a reference. The traffic simulation is coupled with detailed, physics-based powertrain models of both conventional and electric vehicles, which are validated against chassis dynamometer measurements. By analyzing the complex interactions between traffic light control, the resulting vehicle trajectories and the powertrain components, we show that Reinforcement Learning-based adaptive control can significantly improve the traffic flow, with a 41% increase in average velocity, without any drawbacks in CO2 emission (-1%). Furthermore, we find that maximizing traffic flow and minimizing CO2 emissions are not necessarily contradictory objectives, and identify an increased energy saving potential at low traffic densities. Thus, we prove that adaptive traffic light control can make traffic not only more time-efficient, but also more sustainable.

Predictive cruise control functions designed to reduce the energy consumption of intelligent and automated vehicles require an accurate prediction of the upcoming traffic situation in general and the preceding vehicle in particular. This article presents the implementation of prediction models which do not rely on explicit information of the entire preceding vehicle queue. Using this assumption, prediction algorithms based on Conditional Linear Gauss (CLG) models and Deep Neural Network (DNN) were trained using real-world measurements in an urban setting. The training was conducted with 6.6 hours of driving data specifically collected for this study. The results show that both approaches can provide accurate estimates of a preceding vehicles motion. For the CLG models, a queue-estimation logic was implemented to improve the prediction accuracy. The most accurate prediction could be obtained when DNNs are set up with Long Short-Term Memory (LSTM) layers, capable of modeling time dependencies. The results also show that accurate short and mid term predictions of a vehicles trajectory can be made based primarily on sensor data, even when no or only little data from vehicle-to-vehicle (V2V) communication is available.

Advances in artificial intelligence (AI) have led to its application in many areas of everyday life. In the context of control engineering, reinforcement learning (RL) represents a particularly promising approach as it is centred around the idea of allowing an agent to freely interact with its environment to find an optimal strategy. One of the challenges professionals face when training and deploying RL agents is that the latter often have to run on dedicated embedded devices. This could be to integrate them into an existing toolchain or to satisfy certain performance criteria like real-time constraints. Conventional RL libraries, however, cannot be easily utilised in conjunction with that kind of hardware. In this paper, we present a framework named LExCI, the Learning and Experiencing Cycle Interface, which bridges this gap and provides end-users with a free and open-source tool for training agents on embedded systems using the open-source library RLlib. Its operability is demonstrated with two state-of-the-art RL-algorithms and a rapid control prototyping system.

Developing powertrain control functions is time-consuming and resource-intensive, often leading to sub-optimal solutions. Reinforcement Learning (RL) allows agents to perform complex control tasks with minimal human involvement, but is often confined to simulations due to testing costs and safety concerns. To effectively apply RL in embedded powertrain control, agents must be able to handle real-world scenarios, particularly through direct interaction with real actuators and control systems. Therefore, this research applies Transfer Learning (TL) and X-in-the-Loop (XiL) simulations to develop agents that can seamlessly transition and perform robustly in real-world environments. For transient exhaust gas re-circulation control of an internal combustion engine, the process begins with a computationally inexpensive Model-in-the-Loop (MiL) simulation to select a suitable algorithm, fine-tune hyperparameters, and conduct preliminary training. In the next step, pre-trained agents are transferred to an advanced Hardware-in-the-Loop (HiL) system with real hardware using TL for further training. Compared to agents trained entirely on HiL systems, transferred agents required significantly less real-world training time (up to 5.9 times shorter) while outperforming the series production Engine Control Unit (ECU). The results highlight that for real-world effectiveness, integrating actual hardware into training is essential, reward fine-tuning plays a critical role in optimizing these interactions, and the maturity of the policy significantly influences both training duration and overall performance.

Automotive control functions are becoming increasingly complex and their development is becoming more and more elaborate, leading to a strong need for automated solutions within the development process. Here, reinforcement learning offers a significant potential for function development to generate optimized control functions in an automated manner. Despite its successful deployment in a variety of control tasks, there is still a lack of standard tooling solutions for function development based on reinforcement learning in the automotive industry. To address this gap, we present a flexible framework that couples the conventional development process with an open-source reinforcement learning library. It features modular, physical models for relevant vehicle components, a co-simulation with a microscopic traffic simulation to generate realistic scenarios, and enables distributed and parallelized training. We demonstrate the effectiveness of our proposed method in a feasibility study to learn a control function for automated longitudinal control of an electric vehicle in an urban traffic scenario. The evolved control strategy produces a smooth trajectory with energy savings of up to 14%. The results highlight the great potential of reinforcement learning for automated control function development and prove the effectiveness of the proposed framework.