Chlorophyll content is an important indicator for estimating potato growth. However, there are still some research gaps in the inversion of canopy chlorophyll content using unmanned aerial vehicle (UAV) remote sensing. For example, it faces limitations of the growth cycle, low parameter accuracy, and single feature selection, and there is a lack of efficient and precise systematic research methods. In this study, an improved Adaptive-Forward Feature Selection (AFFS) algorithm was developed by combining remote sensing data and measured data to optimize the input Vegetation Index (VI) variables. Gradient Boosting Machine (GBM) model parameters were optimized using a hybrid strategy improved Elephant Herd Optimization (EHO) algorithm (CDE-EHO) that combines Differential Evolution (DE) and Cauchy Mutation (CM). The CDE-EHO method optimizes the GBM model, achieving maximum accuracy, according to the testing results. The optimal coefficients of determination (R²) values of the prediction set are 0.663, 0.683, and 0.906, respectively, the Root Mean Squared Error (RMSE) values are 2.673, 3.218, and 2.480, respectively, and the Mean Absolute Error (MAE) values are 2.052, 2.732, and 1.928, respectively, during the seedling stage, tuber expansion stage and cross-growth stage. This approach has significantly enhanced the inversion model’s prediction performance as compared to earlier research. The chlorophyll content in the potato canopy has been accurately extracted in this work, offering fresh perspectives and sources for further research in this area.