The aim of this study was to examine the association between toothbrushing frequency and school refusal among elementary school children. We used data from the Adachi Child Health Impact of Living Difficulty (A-CHILD) longitudinal study conducted between 2015 and 2016 in Adachi City, Tokyo, Japan. A questionnaire was distributed to all first-grade children aged 6 to 7 years (N = 3697, follow-up rate: 86.2%). Propensity score (PS) matching was applied to collapse the known covariates on toothbrushing frequency in grade 1 on the association with school refusal in grade 2. Among the followed children, 2.4% showed school refusal in grade 2 (89 children) and 23.5% (870 children) brushed their teeth once or less than once daily in grade 1. After propensity score matching, children with toothbrushing once or less than once daily in grade 1 were 2.25 (95% CI: 1.25–4.05) times more likely to show school refusal in grade 2, compared with those with toothbrushing twice or more a day. Our findings suggest that toothbrushing once or less than once daily is an independent risk factor for school refusal among children. Oral health promotion to recommend toothbrushing more than once a day could prevent school refusal. Further intervention studies investigating the mechanism and causality are warranted.