Addressing the thermal comfort of the elderly is a pressing concern amidst the escalating aging population in rural China. This study presents a comprehensive assessment of the thermal comfort within traditional and self-built dwellings in the Guanzhong region. Utilizing a mixed-methods approach encompassing both on-site measurements and questionnaire surveys, with a novel approach of segmenting the elderly population into three distinct age groups. The findings indicate that: (1) An average clothing thermal resistance of 1.83 clo among the elderly, which was found to be relatively higher when compared to other areas and positively correlated with age; (2) A thermal neutral temperature of 8.46 °C for traditional dwellings and 10.53 °C for self-built dwellings, both of which were lower than anticipated, indicating a discrepancy between current living conditions and elderly residents’ thermal expectations; (3) A positive correlation between age and the preferred neutral temperature, with the elderly showing a preference for warmer indoor environments. Additionally, we propose targeted strategies to enhance the thermal comfort of the elderly across different age groups and dwelling types. This study refines the thermal comfort evaluation system for the elderly in China’s cold climate regions, offering theoretical insights and practical guidance for the renovation of rural dwellings and the improvement of elderly living standards in the Guanzhong area.