Objectives: The present study aims to investigate the oral health-related quality of life (OHRQoL) among children with special needs (CSN) and children without special care needs (CWSCN) in Saudi Arabia and to explore the association between various factors, including dental caries status, sociodemographic characteristics, and behavioral factors, with OHRQoL. Methods: A cross-sectional descriptive study was conducted. A total of 773 children were examined (257 with CSN and 516 with CWSCN). OHRQoL was assessed using the Modified Oral Health Impact Profile (OHIP). Multivariable logistic regression was used to determine the relationship between the OHIP (mean score) and independent variables. Results: The mean physical impact was 2.5 ± 1.1 and 3.1 ± 1.7 among 6–11 yrs-old and 12–16-yr-old children (p = 0.021), respectively. The mean personal satisfaction score was 3.2 ± 1.7 and 2.4 ± 1.1 among CSN and CWSCN (p = 0.001), respectively. Children with special needs had a 3.11 (95% CI: 1.23–5.21, p = 0.0001) times higher mean OHIP than CWSCN. Male children had a 1.87 (95% CI: 0.12–2.89, p = 0.024) times higher mean OHIP than female children. Children whose parents had primary school or less education had a 1.92 (95% CI: 0.17–3.11, p = 0.029) times higher mean OHIP than those whose parents had intermediate or higher education. Conclusions: The present study showed that children with special needs had a poor OHRQoL with high mean physical impact, pain, and psychological impact scores compared to CWSCN. A strong association was observed between poor OHRQoL and parental education status, poor oral hygiene practices, and use of non-fluoridated toothpaste.