Seed viability fundamentally determines crop establishment, stress resilience, and yield stability in pepper (Capsicum annuum L.), yet conventional assessment remains destructive, labor-intensive, and poorly scalable, while existing spectral learning approaches largely lack physiological interpretability, limiting their reliability for industrial seed quality management. Here, we present ViaNet, a lightweight, interpretable deep hyperspectral classification framework for 1038 naturally aged pepper seeds labeled via standardized 14-day germination tests. ROI-averaged hyperspectral reflectance vectors are modeled as a binary classification task, and ViaNet integrates Successive Projections Algorithm (SPA)-based wavelength sparsification with Efficient Channel Attention (ECA)-driven spectral weighting within a compact 1D-CNN architecture, enabling physiologically grounded feature learning under strict computational constraints. The model achieves recall for germinable seeds (79.75%) and outperforms classical machine learning methods. In addition, ViaNet consistently highlights reproducible spectral bands associated with natural-aging-related biochemical changes as reported in the literature (e.g., carotenoid-related absorption features in the near-UV region). By coupling spectral feature selection with attention-guided wavelength focusing, ViaNet establishes a closed analytical chain from spectral compression to physiologically interpretable inference. This framework balances predictive accuracy, interpretability, and deployability and provides a scalable, non-destructive, and biologically informed paradigm for hyperspectral seed viability assessment.