Telomere (TL) is a biomarker of biological aging, and its shortening is associated with major risk factors for cardiovascular diseases (CVD). This study aimed to identify whether TL is associated with arterial stiffness as reflected by brachial–ankle pulse wave velocity (baPWV). This population-based cross-sectional study involved 1065 individuals in the Iwaki area, Japan. Total TL length and TL G-tail length were measured by hybridization protection assay. The baPWV was measured on the right and left sides using a non-invasive vascular screening device. The associations between TL and baPWV were assessed by multivariate linear regression. Compared with the shortest total TL tertile, the longest total TL group showed a significant decrease in baPWV (lowest vs. highest tertile: adjusted beta: −41.24, 95% confidence interval (CI): −76.81, −5.68). The mean baPWV decreased with a longer TL (TL G-tail length: p trend < 0.001, total TL: p trend < 0.001). TL G-tail and total TL lengths were inversely associated with baPWV, implicating TL shortening in the development of CVD. This study provides evidence of the factors influencing CVD risks at a very early stage when individuals can still take necessary precautions before CVD gives rise to a symptomatic health outcome.