Active tension control in marine cable winch systems is challenged by vessel heave motion under irregular wave excitation, which generates dynamic disturbances that fixed-gain proportional-integral (PI) controllers cannot fully compensate. This study investigates whether a recurrent reinforcement-learning policy can address this limitation by learning an adaptive, temporally informed control strategy from simulation. A controller based on Proximal Policy Optimization with Long Short-Term Memory (PPO-LSTM) is trained in a Unity ML-Agents environment under stochastic Joint North Sea Wave Project (JONSWAP) wave disturbances and compared with a tuned PI baseline over an 8–20 kN operating range. This is the first application of PPO-LSTM to marine winch tension control and the first ablation study isolating the contributions of recurrent memory and wave-phase sensing in this context. The principal benefit of the recurrent architecture is not a reduction in mean tracking error which remains statistically comparable to the PI baseline but a substantial improvement in control consistency: tension-error variability is markedly reduced, particularly at higher setpoints. Both LSTM and wave-phase observations are shown to contribute independently to this improvement. Key limitations include curriculum bias at low setpoints, catastrophic forgetting during fine-tuning, and the simulation-to-reality gap, for which a staged hardware validation pathway is proposed.

Marine cable winch systems used in offshore operations are exposed to substantial tension fluctuations produced by vessel heave motion and irregular ocean-wave disturbances. These fluctuations can compromise operational stability and may contribute to undesirable slack–re-tension cycles, mechanical loading, fatigue, and reduced reliability. Conventional proportional-integral (PI) controllers remain attractive because of their simplicity and dependable steady-state behavior, but fixed-gain feedback control may be less effective when operating conditions and disturbance patterns vary. This study investigates a reinforcement-learning approach based on Proximal Policy Optimization (PPO) combined with Long Short-Term Memory (LSTM) recurrent memory for adaptive marine winch tension control. The controller is trained entirely in a Unity ML-Agents simulation environment containing a nonlinear drum-winch plant and a stochastic JONSWAP irregular-wave disturbance model. The agent uses a nine-dimensional observation vector comprising normalized tension, setpoint, tracking error, drum angular velocity, rope length, motor torque, dominant wave-phase sine and cosine components, and tension rate of change. A continuous normalized motor-torque command is generated at 60 ms decision intervals. Training employs a graduated curriculum over an 8-20 kN setpoint range, with an initial fixed 8 kN phase followed by randomized setpoints. The final reported model is evaluated against a manually tuned PI controller using randomized JONSWAP realizations. Across the combined operating range, the PPO-LSTM controller achieves a mean tension error of 1.724 kN compared with 1.780 kN for the PI controller, corresponding to a 3.1% improvement that is not statistically significant at the 0.05 level. More importantly, the reinforcement-learning controller reduces error standard deviation by 17.3% overall, with reductions of 26.3% and 39.2% in the 12-15 kN and 15-20 kN ranges, respectively. At the 95th percentile of errors, the overall reduction is 6.4%. The results indicate that the main benefit of the recurrent RL architecture is improved consistency and robustness rather than a large reduction in average tracking error. The study also identifies curriculum bias at low setpoints, increased slack incidence in the 8-12 kN range, catastrophic forgetting during targeted high-setpoint fine-tuning, and the simulation-to-reality gap as important limitations. The expanded discussion therefore emphasizes both the potential and the engineering constraints of PPO-LSTM control for marine winch applications.