The long-term behavioural analysis and health assessment of Pigs are essential for intelligent management in modern pig farming. Manual tracking and behaviour analysis for constructing health assessment systems are often subjective, inconsistent, and lack sufficient accuracy. To overcome these challenges, this study proposes a health assessment framework for pigs based on multi-object behaviour tracking and analysis under large-scale pig farming. The proposed framework consists of three modules: an improved ByteTrack-based multi-object tracking (MOT) module, a behaviour statistics and analysis module, and a health assessment module. The pipeline involves using the MOT module to obtain pigs’ behavioural data, followed by the behaviour analysis module and health assessment module to analyse and evaluate the health status of the pigs. Two datasets comprising 18 videos of healthy pigs and 10 videos of unhealthy pigs were created to validate the framework. Experimental results demonstrated that the improved ByteTrack algorithm achieved high performance in MOT metrics, including a High-Order Tracking Accuracy (HOTA) of 74.0%, Multiple Object Tracking Accuracy (MOTA) of 92.2%, Identification F1 Score (IDF1) of 89.4%, and 43 identity switches (IDs). The behaviour statistics derived from these tracking results enabled reliable inputs for the health assessment model, which accurately assesses the health status of each pig. The results demonstrate that the proposed framework provides an effective solution and reliable technical support for pig health monitoring in modern pig farming.