Nonionic poly(ethylene oxide) alkyl ether (C_iE_j) surfactants self-assemble into aggregates of various sizes and shapes above their critical micelle concentration (CMC). Knowledge on solution attributes such as CMC as well as aggregate characteristics is crucial to choose the appropriate surfactant for a given application, e.g., as a micellar solvent system. In this work, we used static and dynamic light scattering to measure the CMC, aggregation number (N_agg), and hydrodynamic radius (R_h) of four different C_iE_j surfactants (C₈E₅, C₈E₆, C₁₀E₆, and C₁₀E₈). We examined the influence of temperature, concentration, and molecular structure on the self-assembly in the vicinity of the CMC. A minimum in the CMC vs temperature curve was identified for all surfactants investigated. Further, extending the hydrophilic and hydrophobic chain lengths leads to an increase and decrease of the CMC, respectively. The size of the aggregates strongly depends on temperature. N_agg and R_h increase with increasing temperature for all surfactants investigated. Additionally, N_agg and R_h both increase with increasing surfactant concentration. The data obtained in this work further improve the understanding of the influence of temperature and molecular structure on the self-assembly of C_iE_j surfactants and will further foster their use in micellar solvent systems.