It is significant for enhancing a container terminal’s competitiveness to utilize digital and intelligent transformation techniques. Minimizing the number of container relocations is beneficial for decreasing the non-production cost in a container terminal. For the calculation problem of exact mathematical methods and performance problem of existing deep learning models, a multi-decoder dynamic attention model (MDDAM) is proposed. MDDAM can train multiple different policies and output different solutions. Because of the constraint of the Kullback-Leibler divergence between decoders, the decoders are forced to output different probability distributions. To represent the bay configuration information more accurately, a stack feature enhancer (SFE) is defined based on five features of the bay configuration. Ablation experiments demonstrate the effectiveness of the SFE. MDDAM is trained using the REINFORCE algorithm. Extensive experiments show that the proposed MDDAM has advantages over traditional integer programming methods when solving large-scale problems, and achieves state-of-the-art results compared with known deep reinforcement learning methods.