With the emerging development of hyperspectral single-pixel imaging (SPI) systems, the trade-off between the simplicity of optical structure and the correction of chromatic aberration is now an essential factor to be considered. To address both issues simultaneously, we propose a chromatic-aberration-corrected hyperspectral single-pixel imaging scheme, which is based on spectral measurement and dispersion correction. Its achromatism feature is evaluated by optical simulations and proof-of-concept experiments. Moreover, to overcome the shortcomings of traditional algorithms, a new adaptive iterative algorithm is employed, which can further optimize image quality. The results demonstrate that both dispersion and noise in our system are significantly reduced. Taking the position coordinate variance as a figure of merit, we have realized an order of magnitude improvement in the lateral chromatic aberration over the spectral range of 400–780 nm compared to that in conventional hyperspectral SPI. Meanwhile, the contrast-to-noise ratio in our system is enhanced on average by 3 dB. To the best of our knowledge, this is the first such demonstration, and the technique presents possibilities for future integrated applications of high spatial/spectral resolution over the entire visible range, and the system has the potential to be scaled down for future integrated applications.