This study analyzed whether sociodemographic factors, health perception, dietary habits, and screen time are related to physical activity (PA) in older people with and without non-communicable chronic diseases (NCDs). This cross-sectional study analyzed data from the 2019 Brazilian National Health Survey; the sample was older adults (≥60 years old; n = 22,726). The outcome of this study was being physically active or inactive during leisure time, and NCD was used as a moderating variable. The correlates investigated were sociodemographic and health-related variables. According to the logistic regression analysis, it was observed that being male had an association only in the group with NCDs (OR = 1.25 (1.05–1.48)), as well as residing in the northeastern region (OR = 1.26 (1.04–1.53)). On the other hand, high levels of education (OR = 4.09 (2.92–5.2); OR = 1.92 (1.48–2.49)) and income (OR = 1.64 (1.09–2.48); OR = 1.86 (1.33–2.60)) were associated with PA in both groups, as well as dietary habits (OR = 1.03 (1.01–1.05); (OR = 1.05 (1.04–1.07)). Advanced age (OR = 0.96 (0.94–0.97); OR = 0.97 (0.96–0.98)) and reporting a regular health perception (OR = 0.53 (0.43–0.66); OR = 0.61 (0.52–0.73)) were factors associated with physical inactivity in both groups. Gender, education, and income were unequally associated with an active lifestyle in both groups, and therefore, barriers to PA may arise.