Hyperspectral image (HSI) clustering has attracted significant attention due to its broad applications in agricultural monitoring, environmental protection, and other fields. However, the integration of high-dimensional spectral and spatial information remains a major challenge, often resulting in unstable clustering and poor generalization under noisy or redundant conditions. To address these challenges, we propose a Joint Spectral–Spatial Representation Learning (JSRL) framework for robust hyperspectral image clustering. We first perform spectral clustering to generate pseudo-labels and guide a residual Graph Attention Network (GAT) that jointly refines pixel-level spectral and spatial features. We then aggregate pixels into superpixels and employ a Variational Graph Autoencoder (VGAE) to learn structure-aware representations, further optimized via a quantum-behaved particle swarm optimization (QPSO) strategy. This hierarchical architecture not only mitigates spectral redundancy and reinforces spatial coherence, but also enables more robust and generalizable clustering across diverse HSI scenarios. Extensive experiments on multiple benchmark HSI datasets demonstrate that JSRL consistently achieves state-of-the-art performance, highlighting its robustness and generalization capability across diverse clustering scenarios.