Autonomous navigation is critical for unlocking the full potential of Unmanned Surface Vehicles (USVs) in complex maritime environments. Deep Reinforcement Learning (DRL) has emerged as a powerful paradigm for developing self-learning control policies, yet the design of reward functions to balance conflicting objectives, particularly fast arrival at the target position and collision avoidance, remains a major challenge. The precise, quantitative impact of reward parameterization on a USV's maneuvering behavior and the inherent performance trade-offs have not been thoroughly investigated. Here we demonstrate that by systematically varying reward function weights within a framework relying on the Proximal Policy Optimization (PPO), it is possible to quantitatively map the trade-off between collision avoidance safety and mission time. Our results, derived from simulations, show that agents trained with balanced reward weights achieve target-reaching success rates exceeding 98\% in dynamic multi-obstacle scenarios. Conversely, configurations that disproportionately penalize obstacle proximity lead to overly cautious behavior and mission failure, with success rates dropping to 22\% due to workspace boundary violations. This work provides a data-driven methodological framework for reward function design and parameter selection in safety-critical robotic applications, moving beyond ad-hoc tuning towards a more structured parameter influence analysis.

Autonomous navigation is critical for unlocking the full potential of Unmanned Surface Vehicles (USVs) in complex maritime environments. Deep Reinforcement Learning (DRL) has emerged as a powerful paradigm for developing self-learning control policies, yet the design of reward functions to balance conflicting objectives, particularly fast arrival at the target position and collision avoidance, remains a major challenge. The precise, quantitative impact of reward parameterization on a USV’s maneuvering behavior and the inherent performance trade-offs have not been thoroughly investigated. Here, we demonstrate that by systematically varying reward function weights within a framework relying on the Proximal Policy Optimization (PPO), it is possible to quantitatively map the trade-off between collision avoidance safety and mission time. Our results, derived from simulations, show that agents trained with balanced reward weights achieve target-reaching success rates exceeding 98% in dynamic multi-obstacle scenarios. Conversely, configurations that disproportionately penalize obstacle proximity lead to overly cautious behavior and mission failure, with success rates dropping to 22% due to workspace boundary violations. This work provides a data-driven methodological framework for reward function design and parameter selection in safety-critical robotic applications, moving beyond ad-hoc tuning towards a more structured parameter influence analysis.

Development of AI-driven automated driving functions requires vast amounts of diverse, high-quality data to ensure road safety and reliability. However, manual collection of real-world data and creation of 3D environments is costly, time-consuming, and hard to scale. Most automatic environment generation methods still rely heavily on manual effort, and only a few are tailored for Advanced Driver Assistance Systems (ADAS) and Automated Driving (AD) training and validation. We propose an automated generative framework that learns real-world features to reconstruct realistic 3D environments from a road definition and two simple parameters for country and area type. Environment generation is structured into three modules - map-based data generation, semantic city generation, and final detailing. The overall framework is validated by training a perception network on a mixed set of real and synthetic data, validating it solely on real data, and comparing performance to assess the practical value of the environments we generated. By constructing a Pareto front over combinations of training set sizes and real-to-synthetic data ratios, we show that our synthetic data can replace up to 90% of real data without significant quality degradation. Our results demonstrate how multi-layered environment generation frameworks enable flexible and scalable data generation for perception tasks while incorporating ground-truth 3D environment data. This reduces reliance on costly field data and supports automated rapid scenario exploration for finding safety-critical edge cases.

The development of AI-driven automated driving functions requires vast amounts of diverse, high-quality data to ensure road safety and reliability. However, both the manual collection of real-world data and creation of 3D environments are costly, time-consuming, and hard to scale. Most automatic environment generation methods still rely heavily on manual effort, and only a few are tailored for Advanced Driver Assistance Systems (ADAS) and Automated Driving Systems (ADS) training and validation. We propose an automated generative framework that learns ground-truth features to reconstruct 3D environments from a road definition and two simple parameters for country and area type. Environment generation is structured into three modules—map-based data generation, semantic city generation, and final detailing. The overall framework is validated by training a perception network on a mixed set of real and synthetic data, validating it solely on real data, and comparing performance to assess the practical value of the environments we generated. By constructing a Pareto front over combinations of training set sizes and real-to-synthetic data ratios, we show that our synthetic data can replace up to $85\%$ of real data without significant quality degradation. Our results demonstrate how multi-layered environment generation frameworks enable flexible and scalable data generation for perception tasks while incorporating ground-truth 3D environment data. This reduces reliance on costly field data and supports automated rapid scenario exploration for finding safety-critical edge cases.

One of the major challenges in autonomous path following for unmanned surface vehicles (USVs) is the impact of stochastic environmental forces—primarily wind, waves and currents—which introduce nonlinearities that affect control models. Conventional strategies often rely on minimizing cross-track error, resulting in a reactive system that corrects heading only after a disturbance has displaced the vessel, potentially leading to oscillatory behavior and reduced precision. Deep Reinforcement Learning (DRL) is successfully used for a wide range of nonlinear control tasks. It has already been shown that robust solutions that can handle disturbances such as sensor noise or changes in system dynamics can be obtained. This study investigates whether an agent, provided it can explicitly observe disturbances, can go beyond simply correcting deviations and autonomously learn the correlation between environmental conditions and necessary counter-forces. We show that integrating the wind vector directly into the agent’s observation space allows a Proximal Policy Optimization (PPO) policy to decouple the environmental cause from the kinematic effect, facilitating drift compensation before significant errors accumulate. By systematically comparing agents trained with randomized wind scenarios, we found that agents that can observe the wind can achieve goal reaching rates of up to 99.0% and reduce the spread of path deviation and velocity in our tested scenarios. Furthermore, our results quantify a distinct Pareto frontier between navigational velocity and tracking precision, demonstrating that explicit disturbance perception improves consistency, although robust implicit training already provides substantial resilience. These findings indicate that augmenting state observations with environmental data enhances the stability of learning-based controllers.