The combination of cyclin-dependent kinase (CDK) 4/6 inhibitors with endocrine therapy is the standard treatment for patients with HR+/HER2− advanced breast cancer. Recently, this combination has also entered the early setting as an adjuvant treatment in patients with HR+/HER2− disease at a high risk of disease recurrence following (neo)adjuvant chemotherapy. Despite their current use in clinical practice, limited data on the potential gonadotoxicity of CDK4/6 inhibitors are available. Hence, fully informed treatment decision making by premenopausal patients concerned about the potential development of premature ovarian insufficiency and infertility with the proposed therapy remains difficult. The cell cycle progression of granulosa and cumulus cells is a critical process for ovarian function, especially for ensuring proper follicular growth and acquiring competence. Due to the pharmacological properties of CDK4/6 inhibitors, there could be a potentially negative impact on ovarian function and fertility in women of reproductive age. This review aims to summarize the role of the cyclin D-CDK4 and CDK6 complexes in the ovary and the potential impact of CDK4/6 inhibition on its physiological processes.