Background: Immunization is a cost-effective way to prevent infectious diseases in children, but parental hesitancy leads to low vaccination rates, leaving children at risk. Caregivers of children with special health care needs are more hesitant about vaccines than those of healthy children. Objective: The aim of the study is to investigate the changes in caregivers’ vaccination hesitation of children with special health care needs before, during, and after the COVID-19 pandemic in China and to identify associated factors for caregivers’ attitudes toward National Immunization Program (NIP) and non-NIP vaccines. Methods: We included 7770 caregivers of children with special health care needs (median age 7.0, IQR 2.4-24.1 months) who visited the Vaccination Consultation Clinic at Children’s Hospital, Zhejiang University School of Medicine (Hangzhou, China) from May 2017 to May 2023. General and clinical information was extracted from the immunization evaluation system for children with special health care needs and medical records. We compared the differences in caregivers’ willingness and hesitation for vaccinating their children across the 3 stages of the COVID-19 pandemic using chi-square tests. Multinomial logistic regression models were used to identify independent variables that were associated with caregivers’ willingness and hesitation toward NIP and non-NIP vaccines. Results: There is a statistically significant difference in caregivers’ vaccine hesitancy before, during, and after the COVID-19 pandemic (P<.05). During the COVID-19 pandemic, the percentages of choosing NIP, alternative non-NIP, and non-NIP vaccines are highest (n=1428, 26%, n=3148, 57.4%, and n=3442, 62.7%, respectively) than those at other 2 stages. In comparison, caregivers’ hesitation toward NIP and non-NIP vaccines is lowest (n=911, 16.6% and n=2045, 37.3%, respectively). Despite the stages of the COVID-19 pandemic, multiple factors, including children’s age and sex, parents’ educational level, comorbidities, and history of allergy, were significantly associated with caregivers’ attitude toward NIP and non-NIP vaccines (P<.05). The profiles of risk factors for hesitancy toward NIP and non-NIP vaccines are different, as indicated by the results from the logistic regression models. Conclusions: This study demonstrated that caregivers’ willingness to vaccinate their children with special health care needs with NIP and non-NIP vaccines was highest during the COVID-19 pandemic in China, and their hesitancy was lowest. Additionally, we have identified multiple factors associated with caregivers’ willingness and hesitancy to vaccinate their children. These findings provide evidence-based support for developing personalized health education strategies.