The upper reaches of Min River (The upper Min River) is located in Southwest China with significant mountain settlements, which are vulnerable to frequent geological hazards. Based on a field investigation, collation of yearbook data, and analysis through the use of SPSS statistical software, a vulnerability evaluation index system of geological hazards was devised. According to the actual field situation and the acquired data of the study area in 2006, 2009, and 2015, 16 indicators were selected as settlement vulnerability evaluation indexes of geological hazards. The indexes included population density, building coverage, and economic density. Based on the comprehensive evaluation model of entropy value, the dynamic change in the settlement vulnerability of geological hazards was analyzed. The results showed that population density, building coverage, economic density, and road density were the factors that affected the settlement vulnerability of geological hazards the most—Wenchuan earthquake caused considerable damage to the upper Min River, making the area the most vulnerable in 2009. However, its vulnerability decreased in 2015, which indicated that postearthquake reconstruction achieved significant results. Thus, the vulnerability has emerged as an important indicator reflecting the safety and healthy development of mountain settlements.