Teachers’ inclusive education competency is an important factor influencing the realization of high-quality inclusive education. This study investigated the current situation of the inclusive education competency of primary and secondary school physical education teachers and analyzed its influencing factors. There were 286 physical education teachers who participated in the study, including 228 males and 58 females by answering Questionnaire on the Professional Competency of Teachers in Compulsory Education. We found the professional attitude dimension score regarding the inclusive education competency of physical education teachers was significantly higher than that of professional knowledge, professional skills and capacity of acquiring support, whereas the score of professional knowledge dimension was significantly lower than that of professional skills and capacity of acquiring support, and the score of professional skills dimension was higher than that of the capacity of acquiring support dimension. There was no significant difference in the inclusive education competency of physical education teachers in terms of gender, school district and study section, whereas there were significant differences in terms of teaching age, whether children with or without disabilities were taught, and the cumulative length of training related to inclusive education. The inclusive education competency of physical education teachers needed to be further improved.