Valvular heart disease (VHD) accounts for a significant portion of cardiovascular diseases worldwide. Early-stage diagnosis is crucial for effective treatment, yet traditional diagnostic methods rely on harmful or costly modalities. Moreover, existing datasets for VHD often suffer from data scarcity and low quality. In response, we propose a novel VHD detection and classification method utilizing phonocardiography data. Our approach features a classification model based on ProbSparse self-attention and a training strategy that combines virtual adversarial training with Bayesian optimization to address data scarcity effectively. Evaluated on the corresponding public dataset, our method demonstrated robust performance in VHD classification, achieving state-of-the-art results compared to existing approaches. Additionally, through ablation studies, we validated the influence of the adapted components, confirming the effectiveness of our method.