Cities of all sizes around the world are facing increasing levels of congestion, which leads to increasing travel time and emissions, and ultimately affects the quality of life. Relevant research suggests that adaptive traffic light control systems can improve the traffic flow, but their impact on energy-efficiency of vehicle propulsion systems is not well understood. In this study, we use Proximal Policy Optimization, a Deep Reinforcement Learning algorithm, to develop an optimized adaptive traffic light control systems that controls three traffic lights simultaneously. For this purpose, we have created a microscopic traffic simulation of the city of Aachen, Germany, calibrated on the basis of traffic measurements, where the actual traffic light schedule, a green-wave, fixed-time control scheme, serves as a reference. The traffic simulation is coupled with detailed, physics-based powertrain models of both conventional and electric vehicles, which are validated against chassis dynamometer measurements. By analyzing the complex interactions between traffic light control, the resulting vehicle trajectories and the powertrain components, we show that Reinforcement Learning-based adaptive control can significantly improve the traffic flow, with a 41% increase in average velocity, without any drawbacks in CO2 emission (-1%). Furthermore, we find that maximizing traffic flow and minimizing CO2 emissions are not necessarily contradictory objectives, and identify an increased energy saving potential at low traffic densities. Thus, we prove that adaptive traffic light control can make traffic not only more time-efficient, but also more sustainable.

Advances in artificial intelligence (AI) have led to its application in many areas of everyday life. In the context of control engineering, reinforcement learning (RL) represents a particularly promising approach as it is centred around the idea of allowing an agent to freely interact with its environment to find an optimal strategy. One of the challenges professionals face when training and deploying RL agents is that the latter often have to run on dedicated embedded devices. This could be to integrate them into an existing toolchain or to satisfy certain performance criteria like real-time constraints. Conventional RL libraries, however, cannot be easily utilised in conjunction with that kind of hardware. In this paper, we present a framework named LExCI, the Learning and Experiencing Cycle Interface, which bridges this gap and provides end-users with a free and open-source tool for training agents on embedded systems using the open-source library RLlib. Its operability is demonstrated with two state-of-the-art RL-algorithms and a rapid control prototyping system.

Development of AI-driven automated driving functions requires vast amounts of diverse, high-quality data to ensure road safety and reliability. However, manual collection of real-world data and creation of 3D environments is costly, time-consuming, and hard to scale. Most automatic environment generation methods still rely heavily on manual effort, and only a few are tailored for Advanced Driver Assistance Systems (ADAS) and Automated Driving (AD) training and validation. We propose an automated generative framework that learns real-world features to reconstruct realistic 3D environments from a road definition and two simple parameters for country and area type. Environment generation is structured into three modules - map-based data generation, semantic city generation, and final detailing. The overall framework is validated by training a perception network on a mixed set of real and synthetic data, validating it solely on real data, and comparing performance to assess the practical value of the environments we generated. By constructing a Pareto front over combinations of training set sizes and real-to-synthetic data ratios, we show that our synthetic data can replace up to 90% of real data without significant quality degradation. Our results demonstrate how multi-layered environment generation frameworks enable flexible and scalable data generation for perception tasks while incorporating ground-truth 3D environment data. This reduces reliance on costly field data and supports automated rapid scenario exploration for finding safety-critical edge cases.

The development of AI-driven automated driving functions requires vast amounts of diverse, high-quality data to ensure road safety and reliability. However, both the manual collection of real-world data and creation of 3D environments are costly, time-consuming, and hard to scale. Most automatic environment generation methods still rely heavily on manual effort, and only a few are tailored for Advanced Driver Assistance Systems (ADAS) and Automated Driving Systems (ADS) training and validation. We propose an automated generative framework that learns ground-truth features to reconstruct 3D environments from a road definition and two simple parameters for country and area type. Environment generation is structured into three modules—map-based data generation, semantic city generation, and final detailing. The overall framework is validated by training a perception network on a mixed set of real and synthetic data, validating it solely on real data, and comparing performance to assess the practical value of the environments we generated. By constructing a Pareto front over combinations of training set sizes and real-to-synthetic data ratios, we show that our synthetic data can replace up to $85\%$ of real data without significant quality degradation. Our results demonstrate how multi-layered environment generation frameworks enable flexible and scalable data generation for perception tasks while incorporating ground-truth 3D environment data. This reduces reliance on costly field data and supports automated rapid scenario exploration for finding safety-critical edge cases.

Accurately estimating electrical machine temperatures, particularly the magnet temperature, remains a challenging task. Data-driven approaches can be a new method if they offer highly accurate temperature predictions with compact models suitable for real-time embedded systems. This paper presents an investigation on thermal neural networks, a data-driven modeling approach. The main goal of this study is to investigate the benefits and drawbacks of this approach. Firstly, the model architecture and the input dataset from simulations in the ANSYS MotorCAD design software are presented, followed by the training routine. The initial analysis evaluates the best number of nodes for the model, already indicating its strong performance, with a maximum absolute error of the permanent magnet temperature of 1.37 K on an unknown validation dataset. The result of this first analysis is the base model with four internal nodes. This study then explores methods to enhance the physical interpretability of this thermal neural network. Firstly, a sparsing technique reduces the model’s parameters by removing connections that lack physical meaning. Secondly, in an experiment the ambient node is eliminated and a virtual node with constant temperature is introduced to isolate the influence of the ambient node. This model with virtual node has a maximum absolute error of the permanent magnet temperature of 2.75 K and offers faster training with fewer parameters than the base model. The paper finally outlines key steps toward creating a more diverse training dataset, ensuring real-world applicability and usability for temperature-based control systems in permanent magnet synchronous machines.

One of the major challenges in autonomous path following for unmanned surface vehicles (USVs) is the impact of stochastic environmental forces—primarily wind, waves and currents—which introduce nonlinearities that affect control models. Conventional strategies often rely on minimizing cross-track error, resulting in a reactive system that corrects heading only after a disturbance has displaced the vessel, potentially leading to oscillatory behavior and reduced precision. Deep Reinforcement Learning (DRL) is successfully used for a wide range of nonlinear control tasks. It has already been shown that robust solutions that can handle disturbances such as sensor noise or changes in system dynamics can be obtained. This study investigates whether an agent, provided it can explicitly observe disturbances, can go beyond simply correcting deviations and autonomously learn the correlation between environmental conditions and necessary counter-forces. We show that integrating the wind vector directly into the agent’s observation space allows a Proximal Policy Optimization (PPO) policy to decouple the environmental cause from the kinematic effect, facilitating drift compensation before significant errors accumulate. By systematically comparing agents trained with randomized wind scenarios, we found that agents that can observe the wind can achieve goal reaching rates of up to 99.0% and reduce the spread of path deviation and velocity in our tested scenarios. Furthermore, our results quantify a distinct Pareto frontier between navigational velocity and tracking precision, demonstrating that explicit disturbance perception improves consistency, although robust implicit training already provides substantial resilience. These findings indicate that augmenting state observations with environmental data enhances the stability of learning-based controllers.