Attention mechanisms are recently deployed in deep learning models for hyperspectral image (HSI) classification. Conventional spectral attentions typically use global pooling to aggregate spatial information, without sufficiently considering the spatial dependencies of the central pixel to be classified and its neighbours. Moreover, the limited training samples with high-dimensional spectral information make deep learning models prone to over-fitting. In view of these, we propose an end-to-end probabilistic neighbourhood pooling-based attention network (PNPAN) for HSI classification. In PNPAN, we divided the feature maps of input HSI cubes into ring-shaped neighbouring regions and probabilistically selected them as pooling regions to compute channel-wise attention. Based on this, we built a spectral attention-based module and a 3-D convolution module to extract spectral-spatial features. Experiments on three benchmark data sets demonstrate that PNPAN achieves promising results for HSI classification with limited training samples.