Spider mites are important pests that cause severe economic damage to cotton. They feed on underside of leaves, piercing the chloroplast-containing cells, resulting in foliar damage and yield reduction. This paper proposed a two-stage classification approach for mite-infestation detection based on machine learning methods. Two cotton fields were selected for study, and the UAV imagery collection and concurrent ground investigation were conducted on July 20-21th, 2017. Mosaicking and geo-registration were performed on the collected multispectral imagery. Support Vector Machine (SVM) was used for scene classification, and a transferred Convolutional Neural Network (CNN) was applied for mite-infestation identification. Experimental results showed that our method outperformed others in terms of accuracy, which demonstrated that our approach has potential in mite-infestation detection using UAV multispectral imagery.