Residential green and blue spaces may be therapeutic for the mental health. However, solid evidence on the linkage between exposure to green and blue spaces and mental health among the elderly in non-Western countries is scarce and limited to exposure metrics based on remote sensing images (i.e., land cover and vegetation indices). Such overhead-view measures may fail to capture how people perceive the environment on the site. This study aimed to compare streetscape metrics derived from street view images with satellite-derived ones for the assessment of green and blue space; and to examine associations between exposure to green and blue spaces as well as geriatric depression in Beijing, China. Questionnaire data on 1190 participants aged 60 or above were analyzed cross-sectionally. Depressive symptoms were assessed through the shortened Geriatric Depression Scale (GDS-15). Streetscape green and blue spaces were extracted from Tencent Street View data by a fully convolutional neural network. Indicators derived from street view images were compared with a satellite-based normalized difference vegetation index (NDVI), a normalized difference water index (NDWI), and those derived from GlobeLand30 land cover data on a neighborhood level. Multilevel regressions with neighborhood-level random effects were fitted to assess correlations between GDS-15 scores and these green and blue spaces exposure metrics. The average cumulative GDS-15 score was 3.4 (i.e., no depressive symptoms). Metrics of green and blue space derived from street view images were not correlated with satellite-based ones. While NDVI was highly correlated with GlobeLand30 green space, NDWI was moderately correlated with GlobeLand30 blue space. Multilevel regressions showed that both street view green and blue spaces were inversely associated with GDS-15 scores and achieved the highest model goodness-of-fit. No significant associations were found with NDVI, NDWI, and GlobeLand30 green and blue space. Our results passed robustness tests. Our findings provide support that street view green and blue spaces are protective against depression for the elderly in China, yet longitudinal confirmation to infer causality is necessary. Street view and satellite-derived green and blue space measures represent different aspects of natural environments. Both street view data and deep learning are valuable tools for automated environmental exposure assessments for health-related studies.