Automated analysis of long-term Holter recordings is important for efficient arrhythmia screening, but most deep learning approaches rely on raw electrocardiographic waveforms. Heart-rate scatter plots provide a compact visual representation of rhythm dynamics and may support image-based preliminary screening. In this retrospective single-centre study, we developed a deep learning framework for multi-label arrhythmia classification from Holter scatter plot images and evaluated clinically oriented label-specific thresholding. Six hundred de-identified patients, with one Holter record per patient, were included. Expert-reviewed rhythm conclusions were mapped to 15 rhythm labels for model development and evaluation. ResNet18-based models were trained using predefined scatter plot crops. The primary dual-branch model trained with weighted binary cross-entropy achieved macro-F1 = 0.738 (95% CI, 0.666–0.774), micro-F1 = 0.808 (0.777–0.838), macro-AUROC = 0.919 (0.895–0.941) and macro-AUPRC = 0.805 (0.758–0.858) using a fixed threshold of 0.5. Clinically oriented thresholds increased mean specificity from 0.844 (0.822–0.865) to 0.885 (0.860–0.909), but reduced macro-F1 to 0.693 (0.628–0.726). Strong discrimination was observed for atrial fibrillation, atrial flutter, premature atrial contraction and premature ventricular contraction. These findings suggest that scatter plot-based deep learning may provide an efficient preliminary screening approach for Holter analysis, although multicentre external validation is required before clinical implementation.