Warehouse Management Systems have been evolving and improving thanks to new Data Intelligence techniques. However, many current optimizations have been applied to specific cases or are in great need of manual interaction. Here is where Reinforcement Learning techniques come into play, providing automatization and adaptability to current optimization policies. In this paper, we present Storehouse, a customizable environment that generalizes the definition of warehouse simulations for Reinforcement Learning. We also validate this environment against state-of-the-art reinforcement learning algorithms and compare these results to human and random policies.

Warehouse operations involve dynamic item flows, fluctuating demand, and heterogeneous layouts, making adaptive decision-making essential for efficient storage and order fulfillment. In this context, reinforcement learning (RL) provides a promising approach for learning adaptive warehouse control policies under stochastic environments. However, evaluating RL-based solutions in real warehouse settings is often costly and time-consuming, motivating the need for realistic and reproducible simulation environments. In this paper, we introduce a configurable warehouse simulation environment modeling stochastic item arrivals, order generation, and internal logistics operations across diverse layouts and workload conditions. Based on this environment, we construct a reproducible experimental testbed composed of multiple scenarios ranging from low-load to highly congested settings. The testbed is publicly released to support reproducible research and comparative evaluation within the research community. We formulate the warehouse management problem as a Markov decision process (MDP) and apply a Maskable Proximal Policy Optimization (Maskable PPO) agent to learn adaptive control policies. The RL-based approach is evaluated across the defined scenarios and compared against heuristic baseline strategies. Experimental results show that the proposed solution achieves performance comparable to a strong greedy first-in, first-out (FIFO) heuristic while improving order fulfillment by up to 13.5 percentage points under challenging workload conditions. These results demonstrate the ability of RL to learn robust warehouse control policies that adaptively optimize performance and maintain operational stability across a wide spectrum of distinct scenarios.