A comfortable outdoor environment, like its indoor counterpart, can significantly enhance the quality of life and improve the physical and mental health of elderly populations. Urban spatial morphology is one of the key factors influencing outdoor environmental performance. To explore the interactions between urban spatial morphology and the outdoor environment for the elderly, this study utilized parametric tools to establish a performance-driven workflow based on a “morphology generation–performance evaluation–morphology optimization” framework. Using survey data from 340 elderly neighborhoods in Beijing, a parametric urban morphology generation model was constructed. The following three optimization objectives were set: maximizing the winter pedestrian Universal Thermal Climate Index (UTCI), minimizing the summer pedestrian UTCI, and maximizing sunlight hours. Multi-objective optimization was conducted using a genetic algorithm, generating a “morphology–performance” dataset. Subsequently, the XGBoost (eXtreme Gradient Boosting) and SHAP (Shapley Additive Explanations) explainable machine learning algorithms were applied to uncover the nonlinear relationships among variables. The results indicate that optimizing spatial morphology significantly enhances environmental performance. For the summer elderly UTCI, the contributing morphological indicators include the Shape Coefficient (SC), Standard Deviation of Building Area (SA), and Deviation of Building Volume (SV), while the inhibitory indicators include the average building height (AH), Average Building Volume (AV), Mean Building Area (MA), and floor–area ratio (FAR). For the winter elderly UTCI, the contributing indicators include the AH, Volume–Area Ratio (VAR), and FAR, while the inhibitory indicators include the SC and porosity (PO). The morphological indicators contributing to sunlight hours are not clearly identified in the model, but the inhibitory indicators for sunlight hours include the AH, MA, and FAR. This study identifies the morphological indicators influencing environmental performance and provides early-stage design strategies for age-friendly neighborhood layouts, reducing the cost of later-stage environmental performance optimization.