This paper presents a systematic exploration of deep reinforcement learning (RL) for portfolio optimization and compares various agent architectures, such as the DQN, DDPG, PPO, and SAC. We evaluate these agents’ performance across multiple market signals, including OHLC price data and technical indicators, while incorporating different rebalancing frequencies and historical window lengths. This study uses six major financial indices and a risk-free asset as the core instruments. Our results show that CNN-based feature extractors, particularly with longer lookback periods, significantly outperform MLP models, providing superior risk-adjusted returns. DQN and DDPG agents consistently surpass market benchmarks, such as the S&P 500, in annualized returns. However, continuous rebalancing leads to higher transaction costs and slippage, making periodic rebalancing a more efficient approach to managing risk. This research offers valuable insights into the adaptability of RL agents to dynamic market conditions, proposing a robust framework for future advancements in financial machine learning.

This study investigates the application of Reinforcement Learning (RL) to optimize intraday operations of hydropower reservoirs. Unlike previous approaches that focus on long-term planning with coarse temporal resolutions and discretized state-action spaces, we propose an RL framework tailored to the Hydropower Reservoirs Intraday Economic Optimization problem. This framework manages continuous state-action spaces while accounting for fine-grained temporal dynamics, including dam-to-turbine delays, gate movement constraints, and power group operations. Our methodology evaluates three distinct action space formulations (continuous, discrete, and adjustments) implemented using modern RL algorithms (A2C, PPO, and SAC). We compare them against both a greedy baseline and Mixed-Integer Linear Programming (MILP) solutions. Experiments on real-world data from a two-reservoir system and a simulated six-reservoir system demonstrate that while MILP achieves superior performance in the smaller system, its performance degrades significantly when scaled to six reservoirs. In contrast, RL agents, particularly those using discrete action spaces and trained with PPO, maintain consistent performance across both configurations, achieving considerable improvements with less than one second of execution time. These results suggest that RL offers a scalable alternative to traditional optimization methods for hydropower operations, particularly in scenarios requiring real-time decision making or involving larger systems.