The risk for neuropsychiatric illnesses has a strong sex bias, and for major depressive disorder (MDD), females show a more than 2-fold greater risk compared to males. Such mood disorders are commonly associated with a dysregulation of the hypothalamo-pituitary-adrenal (HPA) axis. Thus, sex differences in the incidence of MDD may be related with the levels of gonadal steroid hormone in adulthood or during early development as well as with the sex differences in HPA axis function. In rodents, organizational and activational effects of gonadal steroid hormones have been described for the regulation of HPA axis function and, if consistent with humans, this may underlie the increased risk of mood disorders in women. Other developmental factors, such as prenatal stress and prenatal overexposure to glucocorticoids can also impact behaviors and neuroendocrine responses to stress in adulthood and these effects are also reported to occur with sex differences. Similarly, in humans, the clinical benefits of antidepressants are associated with the normalization of the dysregulated HPA axis, and genetic polymorphisms have been found in some genes involved in controlling the stress response. This review examines some potential factors contributing to the sex difference in the risk of affective disorders with a focus on adrenal and gonadal hormones as potential modulators. Genetic and environmental factors that contribute to individual risk for affective disorders are also described. Ultimately, future treatment strategies for depression should consider all of these biological elements in their design.