Understanding farmers’ participation is crucial for achieving an effective impact on rural living environmental governance and promoting sustainable development. Taking Sandu Town in eastern China as a case study, in-depth semi-structured interviews with farmers, village cadres, and town managers were conducted in this study. Then, a conceptual framework incorporating comprehensive factors is presented to analyze the driving factors and mechanisms of farmer participation in rural domestic waste management. The results show that farmers’ participation in pro-environmental actions is a response to an integrated network of both internal and external factors. Life inertia, loss of personal interests, and objective conditions are the barriers to farmers deciding to participate. Meanwhile, environmental awareness can increase farmers’ internal motivations, and factors such as household benefits, social-cultural influences, and appraisal systems, including household possession protection, very low economic costs, better life experiences, demonstrations from society, “following the crowd”, peer pressure, and reward and criticism measures, are the external forces that mobilize farmers to participate in rural environmental governance. Policy recommendations are proposed based on the findings.