Remote sensing imagery, despite its broad applications in helping achieve Sustainable Development Goals and tackle climate change, has not yet benefited from the recent advancements of versatile, task-agnostic vision language models (VLMs). A key reason is that the large-scale, semantically diverse image-text dataset required for developing VLMs is still absent for remote sensing images. Unlike natural images, remote sensing images and their associated text descriptions cannot be efficiently collected from the public Internet at scale. In this work, we bridge this gap by using geo-coordinates to automatically connect open, unlabeled remote sensing images with rich semantics covered in OpenStreetMap, and thus construct SkyScript, a comprehensive vision-language dataset for remote sensing images, comprising 2.6 million image-text pairs covering 29K distinct semantic tags. With continual pre-training on this dataset, we obtain a VLM that surpasses baseline models with a 6.2% average accuracy gain in zero-shot scene classification across seven benchmark datasets. It also demonstrates the ability of zero-shot transfer for fine-grained object attribute classification and cross-modal retrieval. We hope this dataset can support the advancement of VLMs for various multi-modal tasks in remote sensing, such as open-vocabulary classification, retrieval, captioning, and text-to-image synthesis.

Extreme weather events such as hurricanes and floods cause increasing damage to communities, leading to substantial economic losses and displacement of populations^1,2,3,4,5,6. Previous research suggests that there are disparities in the resilience capacity of neighbourhoods, predicting a recovery mechanism of either segmented withdrawal or reinforcement across different neighbourhood groups^{7,8,9,10,11,12}. Assessing these hypotheses and investigating if—and to what extent—neighbourhood built environments recover at scale has been difficult because previous measures have relied on aggregated survey data^{1,7,9,10,11,12,13,14}. Here we construct a building-level disaster recovery dataset covering 2,195 census tracts spanning 16 states and across 12 extreme weather events in the USA from 2007 to 2023 using historical street view imagery and multimodal machine learning. Our analysis shows that in the aftermath of extreme weather events, lower-income neighbourhoods are less likely to rebuild and do not return to their pre-disaster state, whereas higher-income areas rebuild and tend to improve compared with their pre-disaster state, highlighting increasing disparities in their built environments. We further investigate those disparities by examining the deployment of disaster recovery assistance and insurance policies, and identify a resource gap for lower-income neighbourhoods that may explain unequal community responses to extreme weather events. Our findings demonstrate the value of analysing neighbourhood recovery trajectories at a higher resolution and larger scale to inform responsive policy designs, and suggest the importance of restructuring the recovery financial assistance framework to promote more climate resilient communities.

Urban transformations have profound societal impact on both individuals and communities at large. Accurately assessing these shifts is essential for understanding their underlying causes and ensuring sustainable urban planning. Traditional measurements often encounter constraints in spatial and temporal granularity, failing to capture real-time physical changes. While street view imagery, capturing the heartbeat of urban spaces in a pedestrian point of view, can add as a high-definition, up-to-date, and on-the-ground visual proxy of urban change. We curate the largest street view time series dataset to date, and propose an end-to-end change detection model to effectively capture physical alterations in the built environment at scale. We demonstrate the effectiveness of our proposed method by benchmark comparisons with previous literature and implementing it at the city-wide level. Our approach has the potential to supplement existing dataset and serve as a fine-grained and accurate assessment of urban change.

Accurate, scalable forestry insights are critical for implementing carbon credit-based reforestation initiatives and data-driven ecosystem management. However, existing forest quantification methods face significant challenges: hand measurement is labor-intensive, time-consuming, and difficult to trust; satellite imagery is not accurate enough; and airborne LiDAR remains prohibitively expensive at scale. In this work, we introduce ForestSplat: an accurate and scalable reforestation monitoring, reporting, and verification (MRV) system built from consumer-grade drone footage and 3D Gaussian Splatting. To evaluate the performance of our approach, we map and reconstruct a 200-acre mangrove restoration project in the Jobos Bay National Estuarine Research Reserve. ForestSplat produces an average mean absolute error (MAE) of 0.17 m and mean error (ME) of 0.007 m compared to canopy height maps derived from airborne LiDAR scans, using 100× cheaper hardware. We hope that our proposed framework can support the advancement of accurate and scalable forestry modeling with consumer-grade drones and computer vision, facilitating a new gold standard for reforestation MRV.

Remote sensing imagery, despite its broad applications in helping achieve Sustainable Development Goals and tackle climate change, has not yet benefited from the recent advancements of versatile, task-agnostic vision language models (VLMs). A key reason is that the large-scale, semantically diverse image-text dataset required for developing VLMs is still absent for remote sensing images. Unlike natural images, remote sensing images and their associated text descriptions cannot be efficiently collected from the public Internet at scale. In this work, we bridge this gap by using geo-coordinates to automatically connect open, unlabeled remote sensing images with rich semantics covered in OpenStreetMap, and thus construct SkyScript, a comprehensive vision-language dataset for remote sensing images, comprising 2.6 million image-text pairs covering 29K distinct semantic tags. With continual pre-training on this dataset, we obtain a VLM that surpasses baseline models with a 6.2% average accuracy gain in zero-shot scene classification across seven benchmark datasets. It also demonstrates the ability of zero-shot transfer for fine-grained object attribute classification and cross-modal retrieval. We hope this dataset can support the advancement of VLMs for various multi-modal tasks in remote sensing, such as open-vocabulary classification, retrieval, captioning, and text-to-image synthesis.