Objective: The aim was to investigate the association between baseline coexistence of hypertension (HTN) and arthritis, HTN alone, or arthritis alone, and their impact on longitudinal physical function measures among community-dwelling older adults over 5 years of follow-up. Methods: Ours was a longitudinal prospective cohort study from the second wave (2010–2011) and third wave (2015–2016) of the National Social Life, Health, and Aging Project (NSHAP). Data for older adults were used. Participants were categorized based on self-reported diagnoses into four groups: coexisting HTN and arthritis, HTN only, arthritis only, or neither. Physical performance measures included walking speed using the 3-Meter Walk Test and the Five Times Sit-to-Stand Test (FTSST). Multiple generalized estimating equations with linear regression analyses were conducted, adjusting for age, sex, race, body mass index (BMI) educational level, pain severity, and baseline use of pain and hypertension medications. Results: Data for 1769 participants were analyzed. Slower walking speed was only associated with coexisting HTN and arthritis (B = −0.43, p < 0.001) after accounting for covariates. The coexisting HTN and arthritis group showed no significant association with FTSST (B = 0.80, p = 0.072) after accounting for covariates. Conclusions: The coexistence of baseline HTN and arthritis in older adults is associated with a gradual decline in only walking speed as a physical performance measure in older adults.