Objectives: To develop a diffusion-weighted imaging (DWI) based radiomic signature for predicting early recurrence (ER) (i.e., recurrence within 1 year after surgery), and to explore the potential value for individualized adjuvant chemotherapy. Methods: A total of 124 patients with intrahepatic cholangiocarcinoma (ICC) were randomly divided into the training (n = 87) and the validation set (n = 37). Radiomic signature was built using radiomic features extracted from DWI with random forest. An integrated radiomic nomogram was constructed with multivariate logistic regression analysis to demonstrate the incremental value of the radiomic signature beyond clinicopathological-radiographic factors. A clinicopathological-radiographic (CPR) model was constructed as a reference. Results: The radiomic signature showed a comparable discrimination performance for predicting ER to CPR model in the validation set (AUC, 0.753 vs. 0.621, p = 0.274). Integrating the radiomic signature with clinicopathological-radiographic factors further improved prediction performance compared with CPR model, with an AUC of 0.821 (95%CI 0.684–0.959) in the validation set (p = 0.01). The radiomic signature succeeded to stratify patients into distinct survival outcomes according to their risk index of ER, and remained an independent prognostic factor in multivariable analysis (disease-free survival (DFS), p < 0.0001; overall survival (OS), p = 0.029). Furthermore, adjuvant chemotherapy improved prognosis in high-risk patients defined by the radiomic signature (DFS, p = 0.029; OS, p = 0.088) and defined by the nomogram (DFS, p = 0.031; OS, p = 0.023), whereas poor chemotherapy efficacy was detected in low-risk patients. Conclusions: The preoperative DWI-based radiomic signature could improve prognostic prediction and help to identify ICC patients who may benefit from postoperative adjuvant chemotherapy.