Previous studies suggest an impact of body composition on outcome in melanoma patients. We aimed to determine the prognostic value of CT-based body composition assessment in patients receiving immune checkpoint inhibitor therapy for treatment of metastatic disease using a deep learning approach. One hundred seven patients with staging CT examinations prior to initiation of checkpoint inhibition between January 2013 and August 2019 were retrospectively evaluated. Using an automated deep learning-based body composition analysis pipeline, parameters for estimation of skeletal muscle mass (skeletal muscle index, SMI) and adipose tissue compartments (visceral adipose tissue index, VAI; subcutaneous adipose tissue index, SAI) were derived from staging CT. The cohort was binarized according to gender-specific median cut-off values. Patients below the median were defined as having low SMI, VAI, or SAI, respectively. The impact on outcome was assessed using the Kaplan–Meier method with log-rank tests. A multivariable logistic regression model was built to test the impact of body composition parameters on 3-year mortality. Patients with low SMI displayed significantly increased 1-year (25% versus 9%, p = 0.035), 2-year (32% versus 13%, p = 0.017), and 3-year mortality (38% versus 19%, p = 0.016). No significant differences with regard to adipose tissue compartments were observed (3-year mortality: VAI, p = 0.448; SAI, p = 0.731). On multivariable analysis, low SMI (hazard ratio (HR), 2.245; 95% confidence interval (CI), 1.005–5.017; p = 0.049), neutrophil-to-lymphocyte ratio (HR, 1.170; 95% CI, 1.076–1.273; p < 0.001), and Karnofsky index (HR, 0.965; 95% CI, 0.945–0.985; p = 0.001) remained as significant predictors of 3-year mortality. Lowered skeletal muscle index as an indicator of sarcopenia was associated with worse outcome in patients with metastatic melanoma receiving immune checkpoint inhibitor therapy.