Gestational diabetes mellitus (GDM), an important public health problem that affects mothers and offspring, is a common metabolic disorder. We evaluated the effect of the pre-pregnancy Mediterranean diet (MD) level of exposure on the odds of GDM development. A case-control study (291 GDM cases and 1175 controls without GDM) was conducted in pregnant women. Pre-pregnancy dietary intake was assessed using a validated food frequency questionnaire to calculate an MD adherence index (range score 0–9: low ≤ 2; middle 3–4; high 5–6; very high ≥ 7). Adjusted odds ratios (aOR) and their 95% confidence intervals (CI) were estimated using multivariable logistic regression models including age, BMI, family history of diabetes mellitus, previous GDM, miscarriages, and gravidity. Overall, middle-high MD adherence was 216/291 (74.2%) and very high adherence was 17/291 (5.8%) in cases. In controls the corresponding figures were 900/1175 (76.6%) and 73/1175 (6.2%), respectively. Compared to low adherence, high MD adherence was associated with GDM reduction (aOR 0.61, 95% CI 0.39,0.94; p = 0.028), and very high MD adherence was even more strongly associated (aOR 0.33, 95% CI 0.15, 0.72; p = 0.005). The protective effect of adherence to the MD prior to pregnancy should be considered as a preventive tool against the development of GDM.