(1) Background: Female sexual dysfunction (FSD) has a high prevalence globally, and perinatal factors favor FSD, especially in the postpartum period. The aim was to determine the prevalence and factors influencing FSD in the postpartum period; (2) Methods: An observational study carried out in three primary care centers in southern Spain, with women in the postpartum period who had a single low-risk birth. One hundred and seventeen women answered the Female Sexual Function questionnaire during the 4th month postpartum, between January 2020 and December 2021. Sociodemographic, obstetric, neonatal variables and level of self-esteem were analyzed. A multiple logistic regression model was carried out; (3) Results: 78.4% had high level of self-esteem. FSD prevalence was 89.7%. Factors related to FSD were having an instrumental vaginal delivery, women with university studies, and prenatal preparation. Maternal age ≥ 35, multiparity, pathological processes in the child, a medium–low level of self-esteem and newborn weight were associated with disorders in some of domains of sexual function; (4) Conclusions: FSD is highly prevalent in the postpartum period and is associated with preventable factors. A preventive approach by health professionals to these factors is essential. Health services should implement postpartum follow-up programs, which may coincide in time and place with newborn follow-up programs.