Accurate mapping of crop residue cover (CRC) is important for sustainable agricultural management because residue supports soil health, erosion control, and carbon sequestration. Traditional ground-based CRC methods are labor-intensive and spatially limited, increasing interest in UAV-based remote sensing. UAVs can carry RGB, multispectral, and hyperspectral sensors, which capture different spectral information for distinguishing crop residue from soil. This study compared four high-spatial-resolution (2.5 cm) UAV imagery types—RGB, multispectral, visible–near-infrared (VNIR) hyperspectral, and shortwave infrared (SWIR) hyperspectral—for fine-scale CRC classification. A fully connected neural network (FCNN) was developed to classify residue and soil pixels. Performance was evaluated using two complementary approaches: pixel-level accuracy assessment based on manually delineated image samples and plot-level validation against residue percentages derived from ground photos. Results showed that high pixel-level classification accuracy values were achieved across all imagery types, with overall accuracies above 94%. However, plot-level validation revealed that sensor performance depended on the evaluation metric considered. Multispectral imagery produced the highest R² with ground photo-derived reference CRC values (R² = 0.672). These results indicate that greater spectral dimensionality did not necessarily improve plot-level CRC estimation under the tested field conditions. More importantly, the findings show that high pixel-level classification accuracy does not necessarily translate into stronger plot-level CRC estimation, highlighting the importance of using complementary validation approaches when evaluating UAV-based CRC estimation.