Background: Malnutrition, loss of weight and of skeletal muscle mass are frequent in pancreatic cancer patients, a majority of which will undergo chemotherapy over the course of their disease. Available data suggest a negative prognostic role of these changes in body composition on disease outcomes; however, it is unclear whether tolerance to chemotherapeutic treatment is similarly and/or negatively affected. We aimed to explore this association by retrospectively assessing changes in body composition and chemotherapy-related toxicity in a cohort of advanced pancreatic cancer patients. Methods: Body composition was evaluated through clinical parameters and through radiological assessment of muscle mass, skeletal muscle area, skeletal muscle index and skeletal muscle density; and an assessment of fat distribution by subcutaneous adipose tissue and visceral adipose tissue. We performed descriptive statistics, pre/post chemotherapy comparisons and uni- and multivariate analyses to assess the relation between changes in body composition and toxicity. Results: Toxicity risk increased with an increase of skeletal muscle index (OR: 1.03) and body mass index (OR: 1.07), whereas it decreased with an increase in skeletal muscle density (OR: 0.96). Multivariate analyses confirmed a reduction in the risk of toxicity only with an increase in skeletal muscle density (OR: 0.96). Conclusions: This study suggests that the retrospective analysis of changes in body composition is unlikely to be useful to predict toxicity to gemcitabine—nab-paclitaxel.