Polymorphisms in sex steroid receptors have been associated with transsexualism. However, published replication studies have yielded inconsistent findings, possibly because of a limited sample size and/or the heterogeneity of the transsexual population with respect to the onset of dysphoria and sexual orientation. We assessed the role of androgen receptor (AR), estrogen receptors alpha (ERα) and beta (ERβ), and aromatase (CYP19A1) in two large and homogeneous transsexual male-to-female (MtF) and female-to-male (FtM) populations. The association of each polymorphism with transsexualism was studied with a twofold subject-control analysis: in a homogeneous population of 549 early onset androphilic MtF transsexuals versus 728 male controls, and 425 gynephilic FtMs versus 599 female controls. Associations and interactions were investigated using binary logistic regression. Our data show that specific allele and genotype combinations of ERβ, ERα and AR are implicated in the genetic basis of transsexualism, and that MtF gender development requires AR, which must be accompanied by ERβ. An inverse allele interaction between ERβ and AR is characteristic of the MtF population: when either of these polymorphisms is short, the other is long. ERβ and ERα are also associated with transsexualism in the FtM population although there was no interaction between the polymorphisms. Our data show that ERβ plays a key role in the typical brain differentiation of humans. ERβ plays a key role in human gender differentiation in males and females.