Autonomic nervous system (ANS) dysregulation is an important pathophysiological mechanism in patients with chronic obstructive pulmonary disease (COPD). Heart rate variability (HRV) is a common index for ANS, and HRV has been used to explore the association between ANS and clinical illnesses. This study aimed to explore the group differences in HRV, depression, anxiety, and quality of life between participants with COPD and healthy controls (HC group), and whether emotion plays a mediating role between HRV and quality of life in participants with COPD. A total of ninety-six participants with COPD and 59 participants in the HC group completed the Beck Depression Inventory-II (BDI-II), Beck Anxiety Inventory (BAI), and Saint George’s Respiratory Questionnaire (SGRQ). Assessment of spirometry pulmonary function and five minute lead II electrocardiography (ECG) were also performed under the resting baseline. The COPD group had higher depression scores (F = 4.10, p = 0.008), and a lower quality of life (F = 14.44, p < 0.001) and HRV indices (such as standard deviation of RR intervals (F = 5.49, p < 0.05) and low frequency (F = 3.03, p < 0.05)) compared to the HC group. Sympathetic activation was positively correlated with depression (r = 0.312, p < 0.01), anxiety (r = 0.420, p < 0.001), and poor quality of life (r = 0.467, p < 0.001) in the COPD group. After controlling for age and sex, anxiety (β = 0.585, p < 0.001) and sympathetic activation (β = 0.231, p < 0.05) positively predicted poor quality of life, and lung function (β = −0.251, p < 0.01) negatively predicted poor quality of life. Therefore, anxiety is a mediator between sympathetic activation and quality of life. Emotional and HRV screening should be applied to COPD patients in clinical practice, and emotional management or HRV biofeedback training can be used to improve anxiety and HRV for future studies.