The nature of the association between the number of oocytes retrieved and in vitro fertilization (IVF) outcomes after fresh embryo transfer remains unclear because of conflicting results reported in the studies on this subject. In addition, the influence of the quality of the embryos transferred is usually neglected. The objective of this study is to assess the relationships of the number of oocytes retrieved, the number and quality of embryos transferred, and the prospects of pregnancy after fresh embryo transfer. The data on 3131 infertile women undergoing their first IVF treatment cycle between January 2009 and December 2010 were collected retrospectively. Restricted cubic splines and stratified analyses were used to explore the relationships between the number of oocytes retrieved, the number and quality of embryos transferred, and the IVF outcomes. When stratified by the number and quality of transferred embryos, no significant differences in the chances for clinical pregnancy and live birth were found in three groups of oocytes yielded (≤6, 7–14, or ≥15). The relationship between the number of oocytes retrieved and pregnancy is nearly a reflection of the pattern of the relationship between the number of oocytes retrieved and the probability of having two good-quality embryos transferred. The patients with the “optimal” number of oocytes were not only younger but also had the highest probability of having two good-quality embryos replaced. Similarly aged patients have similar pregnancy prospects after fresh embryo transfer when the same number and quality of embryos are replaced, irrespective of their number of oocytes. Selecting the desired number of good-quality embryos for transfer is the key to IVF success. Thus, aiming at retrieving an optimal number of oocytes to maximize IVF outcomes in a fresh cycle could place undue stress on the patients and may not be the best medical decision.