Granular materials exhibit complex mechanical behaviors during unloading, yet the underlying micro- and meso-scale mechanisms remain unclear. This study employs a discrete element method to simulate a series of triaxial tests on sand and pebble specimens with varying initial densities under different unloading stress paths. While dense specimens demonstrate strain softening and dilatancy, loose samples exhibit shear contraction. To quantify the underlying fabric evolution, persistent homology (PH) theory is adopted to analyze the particle contact force networks. The results reveal that the average strength of this network correlates strongly with the macroscopic stress–strain response. For dense samples, network strength rapidly increases to a peak coinciding with the deviatoric stress maximum, then gradually decreases with further shear. Crucially, this evolution is path-dependent: the average contact force network strength increases approximately 20% more during unloading in the minor principal stress direction compared to unloading in the major principal stress direction. This quantitative analysis of force chain degradation provides a mechanistic explanation for the observed strain softening, highlighting the dominant role of the unloading stress path. In contrast, loose specimens, which initially lack an obvious force chain network, show negligible microstructural evolution during unloading.