Alcohol-drinking levels in Switzerland were investigated to identify dietary differences and explore the relationship between drinking levels and sociodemographic, lifestyle and anthropometric factors using the National Nutrition Survey menuCH (n = 2057, 18–75 years). After two 24 h dietary recalls (24HDRs), participants were categorized into four subgroups: abstainers (both self-declared alcohol avoidance and no alcohol reported); no alcohol reported; moderate drinkers (women/men < 12 g/ 12 g/>24 g mean daily alcohol, respectively). Differences in diet between these groups were described by comparing daily total energy and non-alcohol energy intake, macronutrient energy contribution, food group intake, and diet quality (Alternate Healthy Eating Index excluding alcohol). The sociodemographic, anthropometric and lifestyle factors that determine alcohol-drinking levels were investigated using multinomial logistic regression. Abstainers reported the lowest daily energy intake (total and non-alcohol), heavy drinkers had the highest total energy intake and the lowest diet quality, and moderate drinkers had the highest non-alcohol energy intake. Sex, age, language region, body mass index, household size, smoking status, self-reported health status and following a diet were significantly associated with different alcohol-drinking subgroups. Results could facilitate interventions that target subgroups who exceed safe alcohol-drinking levels and lead unfavorable lifestyles.