Posture transition behavior in sows approaching parturition provides an important physiological cue for farrowing prediction. However, manual monitoring is time-consuming, labor-intensive and difficult to sustain under nighttime production conditions, while existing machine vision approaches remain limited in their ability to represent continuous posture transitions in complex farm environments. Here, we propose an event-level posture transition recognition framework that integrates YOLOv11n with an RGB–Flow cross-modal temporal network. YOLOv11n is first used to detect basic sow postures at the frame level, after which candidate transition events are automatically generated and refined according to temporal state changes. For each event segment, RGB appearance features and optical-flow motion features are extracted to construct dual-branch spatio-temporal representations. We further develop a multi-scale cross-modal attention temporal network (MS-CMATNet) for event-level behavior classification. The network captures local temporal dynamics through a multi-scale module, enhances interactions between RGB and Flow representations through cross-modal attention, and improves feature discriminability and stability by incorporating temporal–channel attention blocks (TCBAM) and an auxiliary cross-modal consistency loss (AuxCross). Experiments show that MS-CMATNet achieves an Accuracy of 88.14%, a Macro-Recall of 84.04%, and a Weighted-F1 score of 87.66% under the fixed training/validation split, outperforming the compared machine learning models, deep temporal models, and representative temporal and cross-modal baselines. Repeated stratified cross-validation and paired t-tests further confirm that MS-CMATNet achieves statistically reliable improvements over most compared baselines, particularly in Macro-F1 and Weighted-F1. These findings demonstrate the potential of the proposed framework for automated farrowing prediction in smart livestock farming.