Plant height and SPAD values are critical indicators for evaluating peanut morphological development, photosynthetic efficiency, and yield optimization. Recent unmanned aerial vehicle (UAV) technology advancements have enabled high-throughput phenotyping at field scales. As a globally strategic oilseed crop, peanut plays a vital role in ensuring food and edible oil security. This study aimed to develop an optimized estimation framework for peanut plant height and SPAD values through machine learning-driven integration of UAV multi-source data while evaluating model generalizability across temporal and spatial domains. Multispectral UAV and ground data were collected across four growth stages (2023–2024). Using spectral indices and Texture features, four models (PLSR, SVM, ANN, RFR) were trained on 2024 data and independently validated with 2023 datasets. The ensemble machine learning models (RFR) significantly enhanced estimation accuracy (R² improvement: 3.1–34.5%) and robustness compared to the linear model (PLSR). Feature stability analysis revealed that combined spectral-textural features outperformed single-feature approaches. The SVM model achieved superior plant height prediction (R² = 0.912, RMSE = 2.14 cm), while RFR optimally estimated SPAD values (R² = 0.530, RMSE = 3.87) across heterogeneous field conditions. This UAV-based multi-modal integration framework demonstrates significant potential for temporal monitoring of peanut growth dynamics.