Background: This study compares the mental health and psychological response of students with or without hearing loss during the recurrence of the COVID-19 pandemic in Beijing, the capital of China. It explores the relevant factors affecting mental health and provides evidence-driven strategies to reduce adverse psychological impacts during the COVID-19 pandemic. Methods: We used the Chinese version of depression, anxiety, and stress scale 21 (DASS-21) to assess the mental health and the impact of events scale—revised (IES-R) to assess the COVID-19 psychological impact. Results: The students with hearing loss are frustrated with their disability and particularly vulnerable to stress symptoms, but they are highly endurable in mitigating this negative impact on coping with their well-being and responsibilities. They are also more resilient psychologically but less resistant mentally to the pandemic impacts than the students with normal hearing. Their mental and psychological response to the pandemic is associated with more related factors and variables than that of the students with normal hearing is. Conclusions: To safeguard the welfare of society, timely information on the pandemic, essential services for communication disorders, additional assistance and support in mental counseling should be provided to the vulnerable persons with hearing loss that are more susceptible to a public health emergency.