The enrichment of cadmium (Cd) in farmland soil poses serious risks to agricultural safety and remains challenging to remediate. This study evaluated CaAl-layered double hydroxide (CaAl-LDH) as a highly efficient and stable passivator for Cd-contaminated soil. Laboratory adsorption tests demonstrated that Cd²⁺ adsorption on CaAl-LDH followed pseudo-second-order kinetics and the Langmuir model, indicating monolayer chemisorption, with a maximum capacity of 469.48 mg·g⁻¹ at pH 6. The adsorption mechanisms include surface complexation, interlayer anion exchange, dissolution–precipitation, and isomorphic substitution. A three-year field trial in Yongkang City, China showed that CaAl-LDH promoted the transformation of Cd in rhizosphere soil from the ion exchange state (F2) to the residual state (F7) and Fe–Mn oxidized state (F5), reducing the exchangeable Cd content by 26.71%. Consequently, Cd content in rice grains decreased by 68.42% in the first year and remained over 37% lower in the second year, consistently below the national food safety limit. Future research should focus on the optimization of material’s stability and application protocol. The results demonstrate that CaAl-LDH provides a cost-effective and sustainable strategy for the in situ passivation remediation of Cd-contaminated farmland, contributing to food safety and sustainable agriculture.