Background: It is unclear that whether childhood neighborhood relationship is associated with mental health among middle-aged and older adults. To overcome this research gap, this study aimed to investigate the association between childhood neighborhood relationship and mental health among the middle-aged and older adults in China. Methods: The data of this study was sourced from the 2014 and 2015 waves of China Health and Retirement Longitudinal Study. We used ordinary least squares and logit regression models to explore the association between childhood neighborhood relationship and mental health among the middle-aged and older adults in China. Results: The regression results indicate that the middle-aged and older adults who lived in place where neighbors had close-knit relationships at childhood was significantly associated with decreased odds of suffering from depressive symptoms (OR = 0.4259, p < 0.001). Furthermore, compared to the middle-aged and older adults who lived in place where neighbors were not close-knit at childhood, those who lived in place where neighbors were close-knit at childhood had a reduced CES–D score (coefficient = − 2.7822, p < 0.001). Conclusion: This study demonstrates the importance of living in place where neighbors had close-knit relationships at childhood. The integrated interventions, including maintaining close-knit neighborhood relationships and strengthening the construction of community, may be useful to improve mental health.