Whether cognitive decline is related to a higher risk of death independent of the initial cognitive function is inconclusive. Evidence of the association between cognitive decline and mortality among Chinese older people is limited. We aimed to examine whether cognitive decline, assessed by the rate of decrease in the Mini-Mental State Examination (MMSE) score, was associated with mortality independent of initial cognitive function (baseline MMSE score) among Chinese older people. We established two successive and non-overlapping cohorts of older adults nested within the Chinese Longitudinal Healthy Longevity Survey (CLHLS), an ongoing, open, community-based cohort survey conducted every 2–3 years. Cognitive function was measured using the Chinese version of the MMSE. A total of 11,732 older adults who completed two consecutive cognitive function examinations were included and followed for 3 years. A Cox proportional hazards model was used to examine the association of cognitive decline with mortality after adjusting for sociodemographic characteristics, health behaviours, comorbidities and initial cognitive function. The mean age was 82.5 years old, and 44.9% (5264/11732) of participants were men. After adjusting for baseline MMSE scores and other covariates, the rate of change in MMSE scores over 3 years was monotonically and positively associated with subsequent 3-year mortality. Compared to those with stable cognitive function, participants with rapid cognitive decline (decline faster than average, a reduction of MMSE scores > 1.62 points/year) had a 75% higher risk of death (hazard ratio = 1.75, 95% confidence interval 1.57–1.95). The association between cognitive decline and mortality was stronger among relatively younger Chinese older people (aged 65–79 years versus 80 years and over) and those with normal cognitive function at baseline (MMSE scores ≥ 24 versus < 24 points), respectively, but did not differ by cohort and sex. Faster cognitive decline was associated with higher mortality independent of initial cognitive function, especially among those aged 65–79 years and those with normal cognitive function at baseline. The association was consistent across two successive cohorts. Our findings indicate the practical significance of monitoring cognitive change in older adults.