Long-term station-keeping of stratospheric airships is challenged by strong time-varying wind fields, pronounced vertical stratification of wind speed and direction, and limited onboard energy. Existing reinforcement-learning approaches typically rely on instantaneous observations to make reactive decisions and therefore struggle to deliver foresighted control in dynamic environments. This paper proposes a Spatio-Temporal Foresight Reinforcement-Learning framework (STF-RL) that explicitly incorporates future wind information. A Transformer is introduced to model multi-step, multi-altitude forecast wind sequences, and a time–height dual positional encoding is designed to characterize both the temporal evolution and the vertical structure of the wind field. A task-conditioned attention pooling mechanism then extracts the future-wind features most relevant to the current state, which are concatenated with the airship state and fed into an actor–critic network to enable foresighted policy learning. A continuous action space supporting three-dimensional maneuvering is constructed, together with a multi-objective reward that jointly accounts for station-keeping performance, energy consumption and safety. Experimental results show that the proposed method outperforms baseline approaches in station-keeping performance, trajectory stability and energy-utilization efficiency, while exhibiting strong robustness across different wind-field conditions.