In the paper, we find that graphene has a strong dielectric loss, but exhibits very weak attenuation properties to electromagnetic waves due to its high conductivity. As polyaniline nanorods are perpendicularly grown on the surface of graphene by an in situ polymerization process, the electromagnetic absorption properties of the nanocomposite are significantly enhanced. The maximum reflection loss reaches −45.1 dB with a thickness of the absorber of only 2.5 mm. Theoretical simulation in terms of the Cole–Cole dispersion law shows that the Debye relaxation processes in graphene/polyaniline nanorod arrays are improved compared to polyaniline nanorods. The enhanced electromagnetic absorption properties are attributed to the unique structural characteristics and the charge transfer between graphene and polyaniline nanorods. Our results demonstrate that the deposition of other dielectric nanostructures on the surface of graphene sheets is an efficient way to fabricate lightweight materials for strong electromagnetic wave absorbents.