Ru Yu

Ru Yu

Auto-generated profile, based on publicly available information

Ru Yu

Institute of Geospatial Information, Information Engineering University, Zhengzhou, China

Research Focus & Metrics

Academic Career Length

1Year (2026-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
18 March 2026 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Verified
Journal Article
Open access
ZSZhuoheng Song
Zhuoheng Song
BJBingchuan Jiang
Bingchuan Jiang
XWXiao Wang
Xiao Wang
YSYong Sun
Yong Sun
RYRu Yu
Ru Yu
TLTingting Li
Tingting Li
Multimodal large language models (MLLMs) have demonstrated promising potential in addressing real-world vision-language tasks. However, existing MLLMs exhibit relatively limited capabilities in understanding street view images, struggling to achieve multi-level semantic understanding and spatial reasoning for region-level fine-grained inference in street view images, which to some extent limits their potential in geospatial artificial intelligence (GeoAI). This study constructs an instruction fine-tuning dataset to guide MLLMs in learning spatial knowledge and scene features from street-view images, called SVGPT-Instruct. This large-scale dataset for street view understanding encompasses 430K instruction samples and 110K street view images. Ultimately, we propose the StreetViewGPT, an MLLM-based method for street view image scene perception and reasoning. The StreetViewGPT is capable of uniformly handling various street view image tasks, including time estimation (TE), weather recognition (WR), scene classification (SC), spatial distance estimation (SDE), and scene graph generation (SGG). The experimental results show that in the TE, WR, and SC tasks, the accuracy of StreetViewGPT improved by 1.33%, 21.68%, and 18.83% respectively compared to the suboptimal model. In the SDE task, the accuracy of predictions falling within the three error tolerance intervals of [75%, 125%], [50%, 150%], and [25%, 175%] reached 88.3%, 97.7%, and 99.5%, respectively, which improved by 78.2%, 78.7%, and 58.6%. In the SGG task, Recall and Mean Recall reached 16.09% and 6.84%, respectively, which improved by 12.49% and 5.4% compared to Qwen2-VL. It demonstrates that StreetViewGPT possesses multi-level semantic understanding and spatial reasoning capabilities, ranging from global semantic perception at the image level to fine-grained reasoning at the region level. The proposed method enhances the recognition and cognition capabilities for spatial scenes in street view images, offering positive implications for smart city perception and autonomous driving navigation research.