The segmentation of buildings using aerial images and laser data (LIDAR) is a key area of study in computer vision and artificial intelligence. In this paper, we proposed a new deep learning-based framework architecture based on U-Net for the MapAI competition, which required participants to perform two tasks for segmenting buildings. On segmentation task 1, our model achieved an Intersection-over-Union (IoU) score of 0.7551 and a Boundary Intersection-over-Union (BIoU) score of 0.5613. On segmentation task 2, our model achieved an IoU score of 0.8555 and a BIoU score of 0.7127. These results demonstrate that our proposed method achieves competitive IoU and BIoU accuracies in building segmentation.

This study analyses the spatio-temporal pattern of parolees using electronic monitoring, where the developed spatial framework supports the Environmental Criminology concepts such as crime patterns or crime attractive locations. A grid-based solution for spatio-temporal analyses is introduced to ensure the anonymity of the parolees. In order to test these developed concepts, the Istanbul Metropolitan Area was selected as the pilot study area. Following the developed concepts of the Crime Pattern Theory, a spatial framework was designed. A novel grid-based weighted algorithm for the most attractive areas was generated via spatial point-of-interest data and a conducted survey among police officers. The designed framework and the spatio-temporal analyses were carried out for 77 parolees using geostatistical methods. The major findings of the study are (a) 24-hour trajectories of each parolee was monitored; (b) the most attractive grids within the city were defined; and (c) for each parolee, the entrance time to the grid and the time spent within that grid were reported and analyzed. This study is a preliminary study for spatio-temporal detection of parolees’ trajectories, where location-based analyses serve well. This study aims to aid decision-makers to better monitor the parolees and justify the benefits of surveillance.

As artificial intelligence (AI) continues to progress, there is a growing recognition of the significance of multimodal foundation models (MFMs; e.g., GPT-4 Vision (GPT-4V)) that can effectively process and integrate different data modalities, such as visual and textual inputs. In particular, the integration of vision and language has demonstrated enormous potential for many geospatial applications, including navigation, urban planning, and disaster response. To harness this potential, this study proposes a multimodal learning framework wtih geospatial intelligence to develop MFMs enhanced by geospatial AI (GeoAI), which can understand and fuse geospatial knowledge from both visual and language inputs. Within this framework, we design a novel method — spatially explicit contrastive learning — to fine-tune a vision-language foundation model, i.e., Contrastive Language-Image Pretraining (CLIP), aiming to develop GeospatialCLIP tailored for understanding geospatial tasks. As a result, our proposed model extends CLIP’s capabilities to learn geospatial semantics for location-based reasoning and land use understanding in urban areas. Moreover, our work sets a new benchmark for the development of GeoAI MFMs, which are capable of comprehending and interpreting large-scale multimodal datasets (e.g., 180k geospatial image-text pairs worldwide) that are augmented by GPT-4V across various geospatial tasks, including image geo-localization, mixed land use detection, and urban perception prediction. By encoding spatial contextual information into text embeddings via geospatial image reasoning, this spatially explicit approach significantly enriches the model’s geospatial knowledge, enabling a deeper level of geospatial insight unlocked for MFMs. Lastly, we examine spatial-context prompt tuning on GeospatialCLIP, which in turn yields more accurate mixed land use predictions, demonstrating the untapped potential of spatial prompts in addressing complex geospatial challenges.

Street-level geolocalization from images is crucial for a wide range of essential applications and services, such as navigation, location-based recommendations, and urban planning. With the growing popularity of social media data and cameras embedded in smartphones, applying traditional computer vision techniques to localize images has become increasingly challenging, yet highly valuable. This paper introduces a novel approach that integrates open-weight and publicly accessible multimodal large language models with retrieval-augmented generation. The method constructs a vector database using the SigLIP encoder on two large-scale datasets (EMP-16 and OSV-5M). Query images are augmented with prompts containing both similar and dissimilar geolocation information retrieved from this database before being processed by the multimodal large language models. Our approach has demonstrated state-of-the-art performance, achieving higher accuracy compared against three widely used benchmark datasets (IM2GPS, IM2GPS3k, and YFCC4k). Importantly, our solution eliminates the need for expensive fine-tuning or retraining and scales seamlessly to incorporate new data sources. The effectiveness of retrieval-augmented generation-based multimodal large language models in geolocation estimation demonstrated by this paper suggests an alternative path to the traditional methods which rely on the training models from scratch, opening new possibilities for more accessible and scalable solutions in GeoAI.