Plant-based dietary patterns may reduce the risk of dyslipidemia. However, not all plant-based foods are beneficial, and limited data exist for the Chinese population. We investigated the association between different plant-based dietary indices and the risk of dyslipidemia in a Chinese middle-aged and elderly population. The study participants (n = 4096) consisted of adults between 35 and 74 years of age from Xinjiang, China. Dietary consumption of the study participants was evaluated using a semi-quantitative food-frequency questionnaire (FFQ). Three different plant-based dietary indices were calculated using data from dietary surveys, including overall plant-based diet index (PDI), healthy plant-based diet index (hPDI), and unhealthy plant-based diet index (uPDI). Based on these indices, we created an adjusted plant-based diet index (aPDI) based on the Xinjiang population actual dietary behavior and health effects of food. We measured the levels of total cholesterol, triglyceride, LDL-C, and HDL-C in the blood of the study participants. We used multivariable logistic regression and restricted cubic spline to analyze the relationship between plant-based diets and dyslipidemia. The findings showed that 36.6% of the participants had dyslipidemia. Higher PDI adherence was related to lower odds of dyslipidemia (Q3 vs. Q1, OR: 0.780, 95% CI: 0.641–0.949; Q4 vs. Q1, OR: 0.799, 95% CI: 0.659–0.970). High aPDI was related to lower odds of dyslipidemia (Q4 vs. Q1, OR: 0.770, 95% CI: 0.628–0.945; Q5 vs. Q1, OR: 0.748, 95% CI: 0.607–0.921). High scores for PDI, hPDI, and aPDI were all related to a reduced risk of low HDL-C (OR: 0.638, 95% CI: 0.491–0.823; OR: 0.661, 95% CI: 0.502–0.870; OR: 0.580, 95% CI: 0.443–0.758). Conversely, a high uPDI score was associated with an increased risk of low HDL-C (OR: 1.349, 95% CI: 1.046–1.740). There was no non-linear relationship between PDI, hPDI, uPDI, and aPDI and the risk of different types of dyslipidemia. Plant-based dietary indices are related to specific types of dyslipidemia risk. Appropriately increasing the consumption of plant-based foods while improving the quality of plant-based dietary patterns is critical for the prevention of dyslipidemia, especially low HDL-C, in the population.