Traditional feature-based image stitching technologies rely heavily on feature detection quality, often failing to stitch images with few features or low resolution. The learning-based image stitching solutions are rarely studied due to the lack of labeled data, making the supervised methods unreliable. To address the above limitations, we propose an unsupervised deep image stitching framework consisting of two stages: unsupervised coarse image alignment and unsupervised image reconstruction. In the first stage, we design an ablation-based loss to constrain an unsupervised homography network, which is more suitable for large-baseline scenes. Moreover, a transformer layer is introduced to warp the input images in the stitching-domain space. In the second stage, motivated by the insight that the misalignments in pixel-level can be eliminated to a certain extent in feature-level, we design an unsupervised image reconstruction network to eliminate the artifacts from features to pixels. Specifically, the reconstruction network can be implemented by a low-resolution deformation branch and a high-resolution refined branch, learning the deformation rules of image stitching and enhancing the resolution simultaneously. To establish an evaluation benchmark and train the learning framework, a comprehensive real-world image dataset for unsupervised deep image stitching is presented and released. Extensive experiments well demonstrate the superiority of our method over other state-of-the-art solutions. Even compared with the supervised solutions, our image stitching quality is still preferred by users.

Homography estimation is an important task in computer vision applications, such as image stitching, video stabilization, and camera calibration. Traditional homography estimation methods heavily depend on the quantity and distribution of feature correspondences, leading to poor robustness in low-texture scenes. The learning solutions, on the contrary, try to learn robust deep features but demonstrate unsatisfying performance in the scenes with low overlap rates. In this paper, we address these two problems simultaneously by designing a contextual correlation layer (CCL). The CCL can efficiently capture the long-range correlation within feature maps and can be flexibly used in a learning framework. In addition, considering that a single homography can not represent the complex spatial transformation in depth-varying images with parallax, we propose to predict multi-grid homography from global to local. Moreover, we equip our network with a depth perception capability, by introducing a novel depth-aware shape-preserved loss. Extensive experiments demonstrate the superiority of our method over state-of-the-art solutions in the synthetic benchmark dataset and real-world dataset. The codes and models will be available at https://github.com/nie-lang/Multi-Grid-Deep-Homography .

Radial distortion, which severely hinders object detection and semantic recognition, frequently exists in images captured using a wide-angle lens. Correction of this distortion of images is crucial in many computer vision applications. In this study, we present DR-GAN, a conditional generative adversarial network (GAN) for automatic radial distortion rectification (DR). To the best of our knowledge, this is the first end-to-end trainable adversarial framework for radial distortion rectification. DR-GAN trained using the proposed low-to-high perceptual loss learns the mapping relation between different structural images rather than estimating multifarious distortion parameters, while also realizing label-free training and one-stage rectification. As a benefit of one-stage rectification, the proposed method is extremely fast with the completion of rectification in real-time. This is approximately 22 × faster than the state-of-the-art methods. The experimental results show that DR-GAN achieves excellent performance in both quantitative measure (PSNR and SSIM) and visual qualitative appearance.

Recently, learning-based distortion rectification schemes have shown high efficiency. However, most of these methods only focus on a specific camera model with fixed parameters, thus failing to be extended to other models. To avoid such a disadvantage, we propose a model-free distortion rectification framework for the single-shot case, bridged by the distortion distribution map (DDM). Our framework is based on an observation that the pixel-wise distortion information is mathematically regular in a distorted image, despite different models having different types and numbers of distortion parameters. Motivated by this observation, instead of estimating the heterogeneous distortion parameters, we construct a proposed distortion distribution map that intuitively indicates the global distortion features of a distorted image. In addition, we develop a dual-stream feature learning module, benefitting from both the advantages of traditional methods that leverage the local handcrafted feature and learning-based methods that focus on the global semantic feature perception. Due to the sparsity of handcrafted features, we discrete the features into a 2D point map and learn the structure inspired by PointNet. Finally, a multimodal attention fusion module is designed to attentively fuse the local structural and global semantic features, providing the hybrid features for the more reasonable scene recovery. The experimental results demonstrate the excellent generalization ability and more significant performance of our method in both quantitative and qualitative evaluations, compared with the stateof- the-art methods.

As the primary means of transportations in modern society, the automobile is developing toward the trend of intelligence, automation, and comfort. In this paper, we propose a more immersive 3-D surround view covering the automobiles around for advanced driver assistance systems. The 3-D surround view helps drivers to become aware of the driving environment and eliminates visual blind spots. The system first uses four fish-eye lenses mounted around a vehicle to capture images. Then, according to the pattern of image acquisition, camera calibration, image stitching, and scene generation, the 3-D surround driving environment is created. To achieve the real-time and easy-to-handle performance, we only use one image to finish the camera calibration through a special designed checkerboard. Furthermore, in the process of image stitching, a 3-D ship model is built to be the supporter, where texture mapping and image fusion algorithms are utilized to preserve the real texture information. The algorithms used in this system can reduce the computational complexity and improve the stitching efficiency. The fidelity of the surround view is also improved, thereby optimizing the immersion experience of the system under the premise of preserving the information of the surroundings.