China’s Loess Plateau is both the largest and deepest loess deposit in the world, and it has long been one of the most severely eroded areas on Earth. With the implementation of the Grain-for-Green Project in 1999, the Loess Plateau has become the most successful ecological restoration zone, and soil organic carbon (SOC) sequestration has greatly increased. However, little is known about the balance of SOC sequestration and vegetation restoration on the Loess Plateau. Thus, this review focused on the SOC sequestration from vegetation restoration in this region. Firstly, the current situations and principal aspects of vegetation restoration processes were reviewed, and the effects of vegetation restoration on SOC sequestration were summarized. Secondly, based on the new technologies and methods for soil carbon (C) sequestration, the mechanism of soil microbial C sequestration was described from the molecular level of genes, and some management measures for SOC sequestration were summarized. Finally, we pointed out the main directions in C sequestration mechanisms for vegetation restoration depending on the basic process of the C cycle, which should integrate into physics, chemistry, and biology. Overall, this review will help us understand the SOC sequestration function and the ecological benefits of vegetation restoration on the Loess Plateau.