To enable autonomous driving in real-world environments that involve a diverse range of geographic variations and complex traffic regulations, it is essential to investigate Deep Reinforcement Learning (DRL) algorithms capable of policy learning in high-dimensional environments characterized by intricate state–action interactions. In particular, closed-loop experiments, which involve continuous interaction between an agent and their driving environment, serve as a critical framework for improving the practical applicability of DRL algorithms in autonomous driving systems. This study empirically analyzes the capabilities of several representative DRL algorithms—namely DDPG, SAC, TD3, PPO, TQC, and CrossQ—in handling various urban driving scenarios using the CARLA simulator within a closed-loop framework. To evaluate the adaptability of each algorithm to geographical variability and complex traffic laws, scenario-specific reward and penalty functions were carefully designed and incorporated. For a comprehensive performance assessment of the DRL algorithms, we defined several driving performance metrics, including Route Completion, Centerline Deviation Mean, Episode Reward Mean, and Success Rate, which collectively reflect the quality of the driving in terms of its completeness, stability, efficiency, and comfort. Experimental results demonstrate that TQC and SAC, both of which adopt off-policy learning and stochastic policies, achieve superior sample efficiency and learning performances. Notably, the presence of geographically variant features—such as traffic lights, intersections, and roundabouts—and their associated traffic rules within a given town pose significant challenges to driving performance, particularly in terms of Route Completion, Success Rate, and lane-keeping stability. In these challenging scenarios, the TQC algorithm achieved a Route Completion rate of 0.91, substantially outperforming the 0.23 rate observed with DDPG. This performance gap highlights the advantage of approaches like TQC and SAC, which address Q-value overestimation through statistical methods, in improving the robustness and effectiveness of autonomous driving in diverse urban environments.

자율주행 기술은 미래 자동차 산업의 핵심 기술로 연구되면서, 최근 관련 센서, 부품, 플랫폼 및 인공지능 기술을 통해 관련 기술이 급격한 발전을 이루고 있다. 본 논문은 최근 자율주행 기술의 주요 이슈로 대두되는 종단간 학습 기술에 관한 논문으로 인지, 판단, 제어 각 모듈을 단일 통합 신경망으로 대체하는 방식에 대해 알아본다. 우선 전통적인 모듈형 자율주행 시스템의 인지, 판단, 제어의 모듈 구조, 한계점에 대해 살펴보고 종단간 학습 자율주행 시스템의 구조 인 이미테이션 러닝과 강화학습의 주요 기술, 훈련 방법, 입력과 출력 형태, 평가 방법 및 지표에 대해 분석한다. 마지막으로 현재 종단간 학습의 기술 이슈 사항 및 관련 구조에 대해 알아보면서 다양한 연구 사례와 실험 결과에 대해 알아본다.

This work presents SAFE-Q, a safety-aware end-to-end autonomous driving network using CrossQ Deep Reinforcement Learning (DRL) designed for reliable performance under long-tail distribution (LTD) conditions. The architecture integrates a multi-task learning (MTL) perception module, a safety reasoning based local path waypoint (LPW) generator, and a DRL based action planning and control (APC) module. The perception network extracts bird’s eye view (BEV) and perspective view (PV) features from multi-camera inputs, while the safety reasoning module refines global path waypoints to ensure rule compliance and collision avoidance. The DRL module fuses perception and waypoint information within an affordance state vector to produce optimal throttle and brake commands. Experiments in the CARLA simulator reproduced LTD scenarios to evaluate SAFE-Q’s driving performance. SAFE-Q outperformed baseline methods, achieving gains of 12.6 % in Driving Score, 10.1% in Route Completion, 2.4% in Infraction Penalty, 27.9% in Euclidean Distance, 55.3% in Average Speed, and 10% in Jerk. These results confirm SAFE-Q’s safe, efficient, and robust control capability in complex urban LTD environments.