The spatial pattern of color patches plays a crucial role in affecting the visual quality of peri-urban forests dominated by Cotinus coggygria var. cinerea Engl. in autumn. The impact mechanism has been studied to facilitate algorithm-based automatic visual quality estimation. The color patterns of 120 photographs were calculated after color quantization and automatic color substitution. The scenic beauty of the forest was estimated by 698 respondents. Multiple correlations between visual quality and color pattern metrics were explored with stepwise regression. Principal component analysis (PCA) was also employed to investigate the impact mechanism of color patterns on visual quality. Number of patches (NP), largest patch index (LPI), mean patch area (AREA_MN), patch size standard deviation (AREA_SD), and Shannon’s evenness index (SHEI) were the main factors affecting the visual quality of the Cotinus coggygria forest. AREA_MN correlated positively with visual quality, while NP, LPI, AREA_SD, and SHEI correlated negatively. Moreover, AREA_SD had the most significant impact on the visual quality of the landscape, while SHEI, LPI, and AREA_MN had the second-highest impact. The evenness and the size of color patches significantly affected the visual quality of the forest landscapes. Balancing the diversity and evenness of color patches plays a decisive role in creating a forest landscape with high visual quality.