Plant height (PH) and its component traits are critical determinants of lodging resistance and strongly influence yield in rice. The genetic architecture of PH and its component traits were mined in two mapping populations. In the natural population composed of 504 accessions, a total of forty simple sequence repeat (SSR) markers associated with PH and its component traits were detected across two environments via association mapping. Allele RM305-210 bp on chromosome 5 for PH had the largest phenotypic effect value (PEV) (−51.42 cm) with a reducing effect. Allele RM3533-220 bp on chromosome 9 for panicle length and allele RM264-120 bp on chromosome 8 for the length of upper first elongated internode (1IN) showed the highest positive PEV. Among the elongated internodes with negative effects being desirable, the allele RM348-130 bp showed the largest PEV (−7.48 cm) for the length of upper second elongated internode. In the chromosome segment substitution line population consisting of 53 lines, a total of nine QTLs were detected across two environments, with the phenotypic variance explained (PVE) ranging 10.07–28.42%. Among the detected QTLs, q1IN-7 explained the largest PVE (28.42%) for the 1IN, with an additive of 5.31 cm. The favorable allele RM257-125 bp on chromosome 9 for the 1IN increasing was detected in both populations. The favorable alleles provided here could be used to shape PH architecture against lodging.