Activated hepatic stellate cells (aHSCs) are the main orchestrators of the fibrotic cascade in inflamed livers, with transforming growth factor-beta (TGF-β) being the most potent pro-fibrotic cytokine. Hence, aHSCs serve as interesting therapeutic targets. However, drug delivery to aHSCs is hindered by excessive collagen deposition in the extracellular matrix (ECM) and capillarization of liver sinusoids. Chitosan-nanoparticles (CS-NPs) show intrinsic affinity for collagen, holding potential for drug delivery to fibrotic livers. Here, we employed CS-NPs for anti-TGF-β siRNA delivery, promoting delivery into aHSCs via modification with platelet-derived growth factor receptor-beta binding peptides. In-vitro experiments using aHSCs demonstrated the association of unmodified CS-NPs to the collagen-rich ECM, with reduced intracellular accumulation. Peptide-modified CS-NPs showed a higher propensity to localize intracellularly; however, this was only the case upon ECM-collagen reduction via collagenase treatment. Peptide-modified CS-NPs were more potent than unmodified CS-NPs in reducing TGF-β expression, implying that while collagen binding promotes liver accumulation, it hinders cell-specific siRNA delivery. In-vivo, CS-NPs successfully accumulated in fibrotic livers via collagen binding. Similar to in-vitro findings, when mice were pretreated with collagenase-loaded CS-NPs, the accumulation of peptide-modified NPs increased. Our findings demonstrate the usefulness of NPs modification with targeting ligands and collagenase treatment for aHSCs targeting and highlight the importance of chitosan–collagen binding in drug delivery to fibrotic diseases.