Background/Objectives: Durvalumab plus Tremelimumab (Dur/Tre) therapy is expected to have good therapeutic efficacy for patients with advanced hepatocellular carcinoma (HCC). However, the predictors of clinical response and prognosis have not been established. Here, we retrospectively investigated the predictors for therapeutic response and clinical prognosis of Dur/Tre therapy. Methods: This retrospective single-center study, which included 30 patients, aimed to evaluate predictors of treatment efficacy of Dur/Tre therapy for advanced HCC. Factors associated with an objective response rate (ORR), progression-free survival (PFS), overall survival (OS), and immune-mediated adverse events (imAEs) were examined. We especially focused on the initial change in peripheral lymphocyte count at 2 weeks after Dur/Tre introduction from baseline (Δlymphocyte). Results: Seventeen patients (56.7%) diagnosed with HCC BCLC stage C were enrolled. The median observational period was 11 months. The ORR and disease control rate (DCR) were 30.0% and 53.3%, respectively. The median PFS was 3.7 months and OS was not reached. The high Δlymphocyte was an independent predictor of objective response (hazard ratio [HR], 1.004; p = 0.016). The high Δlymphocyte (above +245/µL) was an independent predictive factor for better PFS (HR, 0.308; 95% CI, 0.095–0.998; p = 0.049), and the median PFS was significantly prolonged in the high Δlymphocyte (above +245/µL) compared to low Δlymphocyte (less than +245/µL) (not reached vs. 1.96 months, log-rank p = 0.003). Conclusions: In Dur/Tre therapy, the early increase in peripheral blood lymphocyte count was useful for predicting clinical response and prognosis.