The vulnerability of ecological environment threatens social and economic development. Recent studies failed to reveal the driving mechanism behind it, and there is little analysis on the spatial clustering characteristics of the vulnerability of urban agglomerations. Therefore, this article estimates ecological environment vulnerability in 2005, 2011, and 2017, determines Moran Index (MI) with spatial autocorrelation model, analyzes the spatial-temporal difference characteristics of ecological environment vulnerability of Yangtze River Urban Agglomeration and the spatial aggregation effect, and discusses its driving factors. The study results estimate that the overall vulnerability index of the Yangtze River Urban Agglomeration is in a mild fragile state. However, most fragile and slightly fragile cities are developing in the direction of moderate to severe vulnerability. The spatial agglomeration effect of the ecological environment vulnerability of the Yangtze River Urban Agglomeration is not obvious, and the effect of mutual ecological environment influence among cities is not obvious. Moreover, the driving factors of ecological environment vulnerability of Yangtze River city group changed from natural factors to social economic factors and then to policy factors. It is necessary to develop an ecological economy, coordinate the spatial agglomeration of urban agglomerations, and make balance the internal differences of urban agglomerations.