This paper aims to tackle the challenge of dynamic view synthesis from multi-view videos. The key observation is that while previous grid-based methods offer consistent rendering, they fall short in capturing appearance details of a complex dynamic scene, a domain where multi-view image-based rendering methods demonstrate the opposite properties. To combine the best of two worlds, we introduce Im4D, a hybrid scene representation that consists of a grid-based geometry representation and a multi-view image-based appearance representation. Specifically, the dynamic geometry is encoded as a 4D density function composed of spatiotemporal feature planes and a small MLP network, which globally models the scene structure and facilitates the rendering consistency. We represent the scene appearance by the original multi-view videos and a network that learns to predict the color of a 3D point from image features, instead of memorizing detailed appearance totally with networks, thereby naturally making the learning of networks easier. Our method is evaluated on five dynamic view synthesis datasets including DyNeRF, ZJU-MoCap, NHR, DNA-Rendering and ENeRF-Outdoor datasets. The results show that Im4D exhibits state-of-the-art performance in rendering quality and can be trained efficiently, while realizing real-time rendering with a speed of 79.8 FPS for 512x512 images, on a single RTX 3090 GPU. The code is available at https://zju3dv.github.io/im4d.

We present a novel method for local image feature matching. Instead of performing image feature detection, description, and matching sequentially, we propose to first establish pixel-wise dense matches at a coarse level and later refine the good matches at a fine level. In contrast to dense methods that use a cost volume to search correspondences, we use self and cross attention layers in Transformer to obtain feature descriptors that are conditioned on both images. The global receptive field provided by Transformer enables our method to produce dense matches in low-texture areas, where feature detectors usually struggle to produce repeatable interest points. The experiments on indoor and outdoor datasets show that LoFTR outperforms state-of-the-art methods by a large margin. We further adapt LoFTR to modern SfM systems and illustrate its application in multiple-view geometry. The proposed method demonstrates superior performance in Image Matching Challenge 2021 and ranks first on two public benchmarks of visual localization among the published methods. The code is available at https://zju3dv.github.io/loftr .

Visual geo-localization empowers unmanned aerial vehicles (UAVs) to navigate in Global Navigation Satellite System (GNSS)-denied environments. This capability is typically formulated as an image retrieval task, where UAV-acquired images are matched against geo-tagged satellite image patches. However, existing datasets are predominantly limited to single urban scenarios and often assume an ideal spatial alignment between UAV and satellite, which is unrealistic in real-world settings. To address these limitations, we present World-UAV, a large-vocabulary dataset that offers: (1) extensive scene diversity, spanning 27 unique geographical categories, and (2) realistic spatial discrepancies, incorporating significant geometric variations in rotation and scale between UAV and satellite image pairs. These geometric transformations pose substantial challenges to current global descriptor-based methods, which exhibit marked performance degradation in such scenarios. We propose UAVPlace, a novel learning approach incorporating geometric transformation encoding modules to integrate multi-perspective transformation features, thereby generating transformation-invariant global descriptors. Extensive experiments demonstrate the effectiveness of our method.

Image matching serves as a core component for UAV localization guided by satellite imagery. However, this task remains highly challenging due to the extreme viewpoint discrepancies between low-altitude UAV images and nadir-view satellite maps. Existing datasets primarily focus on ground-level or high-altitude UAV imagery, lacking sufficient coverage of the geometric variations typical of real aerial scenarios. To address these limitations, we introduce AerialExtreMatch, a large-scale, high-fidelity dataset tailored for extreme-view image matching and UAV localization. It consists of approximately 1.5 million synthetic image pairs rendered from high-quality 3D scenes, simulating diverse UAV and satellite viewpoints to enable robust training of image matching models. To support fine-grained evaluation, we construct a hierarchical benchmark with 32 difficulty levels. These are defined using three geometric criteria: overlap ratio, scale variation, and pitch difference. In addition, we collect a real-world UAV localization dataset with geo-aligned reference maps of varying visual quality. Extensive experiments involving 16 representative detector-based and detector-free methods demonstrate that models trained on AerialExtreMatch achieve substantial performance gains in both image matching and real-world localization under extreme-view conditions.