Precise classification of unsound wheat grains is essential for crop yields and food security, yet most existing approaches rely on vision-only models that demand large labeled datasets, which is often impractical in real-world, data-scarce settings. To address this few-shot challenge, we propose UWGC, a novel vision-language framework designed for few-shot classification of unsound wheat grains. UWGC integrates two core modules: a fine-tuning module based on Adaptive Prior Refinement (APE) and a text prompt enhancement module that incorporates Advancing Textual Prompt (ATPrompt) and the multimodal model Qwen2.5-VL. The synergy between the two modules, leveraging cross-modal semantics, enhances generalization of UWGC in low-data regimes. It is offered in two variants: UWGC-F and UWGC-T, in order to accommodate different practical needs. Across few-shot settings on a public grain dataset, UWGC-F and UWGC-T consistently outperform existing vision-only and vision-language methods, highlighting their potential for unsound wheat grain classification in real-world agriculture.