In adversarial reinforcement learning, designing dense reward functions is a traditional approach to address the sparsity of adversarial objectives. However, conventional reward design often relies on high-quality domain knowledge and may fail in practice, thereby inducing objective misalignment—a discrepancy between optimizing the designed reward and achieving the true adversarial utility. To reduce this discrepancy, a Value-Based Reward Shaping (VBRS) framework is proposed. VBRS integrates an intrinsic state-value estimate, which is a dynamic predictor of long-term utility, into the immediate reward function. As a result, exploration can be encouraged toward states predicted to be strategically advantageous, potentially avoiding some local optima in practice. Experiments demonstrate that VBRS outperforms a baseline that relies solely on the original reward function. The results confirm that the proposed method enhances adversarial performance and helps bridge the gap between designed reward guidance and the adversarial objective.

Reinforcement learning is increasingly deployed in autonomous systems whose reward functions encode operational priorities and decision-making objectives. However, observable behavior can enable external observers to infer sensitive reward parameters through inverse reinforcement learning. Existing deployment-time defenses mainly rely on action randomization or deception, without explicitly accounting for how previously released behavior affects the additional reward-sensitive information contributed by subsequent actions. To address this gap, we propose FisherShield, a deployment-time mechanism that balances task value against marginal reward-information exposure. Under a specified reference observer model, FisherShield computes the sensitivity of action log-likelihoods to the private reward and accumulates the sensitivity directions of released actions in an empirical information matrix. For each candidate action, the increase in the matrix’s log-determinant measures its marginal reference-score exposure and is converted into a weighted penalty subtracted from the task value. Updating the matrix after each release makes the penalty depend on the observation history, enabling sequential exposure control without retraining the task policy. Experiments show that FisherShield reduces cumulative reference-score log-det exposure by 26.4% in FourRooms and 34.3% in LavaGrid relative to Expert while preserving near-expert utility. The results are limited to the specified reference-model Bayesian observer and do not establish robustness to model misspecification or a defense-aware attacker.

High-speed unmanned aerial vehicle (UAV) pursuit-evasion in obstacle-rich environments is a challenging safety-critical decision-making problem, where the pursuer must intercept a maneuvering evader while avoiding obstacles. Existing methods often treat safety as an expected constraint and fail to prevent rare but severe unsafe events, resulting in conservative or aggressive policy. To address this issue, we propose a Risk-Aware Switch-Time Recovery Planning (ReSwitch) framework, which separates task-oriented pursuit from safety-oriented recovery. When flight risk emerges, a value-preserving switch-time planner evaluates candidate switching times through opponent-conditioned hybrid rollouts. The feasible switch time with the largest pursuit value after recovery is selected for execution, allowing the controller to prioritize safety while preserving pursuit effectiveness whenever possible. Experiments demonstrate that ReSwitch improves safety performance over baselines, with less degradation in pursuit effectiveness. The results show that ReSwitch is an effective mechanism for balancing safe flight and task completion.

UAV pursuit–evasion in multi-obstacle environments constitutes a safety-critical decision-making problem in which the pursuer must intercept a maneuvering evader while satisfying obstacle-avoidance constraints. Existing safety-constrained methods typically optimize expected safety costs or apply immediate interventions, but they do not explicitly determine when recovery should begin once finite-horizon risk emerges. To address this issue, we propose the ReSwitch framework, which separates task-oriented pursuit from safety-oriented recovery. When flight risk emerges, a value-preserving switch-time planner evaluates candidate switching times through opponent-conditioned hybrid rollouts. The feasible switch time with the largest pursuit value after recovery is selected for execution, allowing the controller to prioritize safety while preserving pursuit effectiveness whenever possible. Experiments demonstrate that ReSwitch achieves a mean success rate of 89.02% and a physical safety rate of 91.67%, showing favorable performance compared with the baselines. These results indicate that ReSwitch provides a favorable pursuit–safety trade-off under the tested conditions.

Sparse rewards remain a major obstacle to efficient reinforcement learning (RL), particularly in long-horizon goal-reaching tasks where little feedback is available before successful termination. This difficulty is further complicated by time-limited trajectories, for which the absence of observed success indicates incomplete follow-up rather than necessarily a failed attempt. Existing progress-based shaping methods generally do not explicitly model this distinction, which can bias temporal progress estimates and weaken the resulting shaping signal. To address this issue, we propose Censoring-Aware Time-to-Success Potential Shaping (CTPS), a reward-shaping method that treats successful trajectories as observed time-to-event data and time-limited trajectories as right-censored data. CTPS fits a discrete hazard model to estimate the conditional time-to-success distribution, summarizes the resulting survival curve using the restricted mean time-to-success, and converts this estimate into a bounded potential. The learned temporal signal is then incorporated through a potential-based reward difference, with the potential held fixed during each target-update interval. Experiments show that CTPS reduces RMST estimation error by approximately 93% relative to timeout-as-event estimation under heavy censoring, while improving sparse-maze success rates by approximately 157% over NaiveTTS-PBRS. It also preserves state-ranking correlations above 0.98 under the evaluated behavior-policy shifts. These results characterize the temporal-estimation performance of CTPS and its use as a shaping signal in the evaluated sparse-reward tasks.

Adversarial policies provide a practical means of stress-testing reinforcement-learning agents by inducing failure through environment interactions without modifying the deployed victim policy. Learning effective adversarial policies under partial observability is challenging because the attacker must act from incomplete observations without access to victim-side parameters, gradients, or internal states. Existing approaches often rely on instantaneous observability patterns or predefined partitions, which can discard temporal interaction information and provide limited guidance for exploring rarely encountered but potentially vulnerable contexts. We propose CAAP (Context-Adaptive Adversarial Policy), which is a black-box adversarial policy framework that represents recent interaction history with a recurrent Context Encoder, combines multiple action-value experts through a differentiable Soft Policy Router, and encourages early exploration through Context Coverage Exploration using an annealed KNN-based coverage bonus in the learned context space. Context Coverage Exploration is used only during training, leaving the Context Encoder, Soft Policy Router, and action-value experts as the deployed policy. In the primary particle-pursuit benchmark, CAAP achieves the lowest in-domain catch rate among the five internal methods and the lowest macro-average catch rate across eight zero-update transfer conditions. These results support the use of history-dependent Soft Policy Routing with training-only Context Coverage Exploration for black-box adversarial policy learning under partial observability.