Cross-spectral image patch matching is still challenging due to significant nonlinear differences between image patches. Recently, image patch matching methods based on feature relation learning have attracted increasing attention and achieved good performance. However, we find that the metric learning methods based on feature difference cannot comprehensively and effectively extract useful discriminative information between image patch pairs by only adopting the two-branch network structure. Therefore, we propose a novel multibranch feature difference learning network (MFD-Net). Specifically, we build a multibranch parallel feature difference extraction network, which can capture richer and more discriminative feature difference information and achieve significant improvements on matching tasks. Furthermore, we propose a combined metric network composed of a master metric network module and multiple branch metric network modules, which promotes the forward update of network weights and reduces the similarity of features extracted by each feature difference extraction module with a negligible increase in inference time. Extensive experimental results show that the proposed MFD-Net achieves superior performances on cross-spectral image patch matching and single spectral image patch matching.

This paper considers the joint geometric and photometric image registration problem. The inverse compositional (IC) algorithm and the efficient second-order minimization (ESM) algorithm are two typical efficient methods applied to the geometric registration problem. Their efficiency stems from the utilization of the group structure of geometric transformations. To allow for photometric variations, the dual IC algorithm (DIC) proposed by Bartoli performs joint geometric and photometric image registration by extending the IC algorithm. The group structures of both geometric and photometric transformations are exploited. Despite the robustness to large photometric variations, DIC is vulnerable to large geometric deformations. The ESM algorithm is extended by Silveira et al. to address photometric variations. In their approach, the photometric transformations are modeled in Euclidean space. Their approach is robust to relatively large geometric and photometric transformations; however, it is not efficient for large photometric variations. We propose a new efficient and robust image registration method by exploiting the non-Euclidean Lie group structure of joint geometric and photometric transformations for both grayscale and color images. The image registration is formulated as a nonlinear least squares problem. In our method, the geometric and photometric transformations are jointly parameterized by their corresponding Lie algebras. Based on this parameterization approach, the second-order approximation strategy of ESM is employed to optimize the joint geometric and photometric parameters. The error function in the nonlinear least squares problem is approximated by a second-order Taylor expansion with respect to joint geometric and photometric parameters without computing the Hessian matrix. For further efficiency, independent convergence criteria for geometric and photometric parameters are used in the iterative optimization process. The superiority of our proposed method over the previous methods, in terms of efficiency, accuracy, and robustness, is demonstrated through extensive experiments on synthetic and real data.

This letter proposes a generalized framework with joint normalization that learns lower dimensional subspaces with maximum discriminative power by using Riemannian geometry. We model the similarity/dissimilarity between subspaces using various metrics defined on Grassmannian and formulate dimensionality reduction as a nonlinear constraint optimization problem considering the orthogonalization. To obtain the linear mapping, we derive the components required to perform Riemannian optimization from the original Grassmannian through an orthonormal projection. We respect the Riemannian geometry of the Grassmann manifold and search for this projection directly from one Grassmann manifold to another face-to-face without any additional transformations. In this natural geometry-aware approach, any metric on the Grassmann manifold can theoretically reside in our model. We combine five metrics with our model, and the learning process is treated as an unconstrained optimization problem on a Grassmann manifold. Experiments on several datasets demonstrate that our approach leads to a significant accuracy gain over state-of-the-art methods.

Geometric transformations of images are the predominant factor, which influences the effectiveness of visual tracking and detection tasks in computer vision. Naturally, although it makes significant sense to grasp the process of image geometric transformations, the numerical relationship of geometric transformations cannot be revealed directly from images themselves. Even if the geometric transformation matrices form the three-dimensional special linear group, Sl(3, ℝ) group, it is difficult to comprehend the manifold of this invisible visual motion, which resides in the high-dimensional space. Furthermore, the main challenge is the deficiency of analytic expressions of the Riemannian logarithmic map to compute the geodesic distance on the Sl(3, ℝ) manifold. Facing these issues, this paper comes up with a novel approach to visualize the geometric transformation in images by presenting a new metric, and then, computes a set of coordinate-vectors in the three-dimensional state transition space for visualization using the Riemannian stress majorization. The superiority of the presented framework for visualization, in terms of accuracy and efficiency, is demonstrated through abundant experiments on aerial images and moving objects.

Modeling videos and image sets by linear subspaces has achieved great success in various visual recognition tasks. However, subspaces constructed from visual data are always notoriously embedded in a high-dimensional ambient space, which limits the applicability of existing techniques. This letter explores the possibility of proposing a geometry-aware framework for constructing lower-dimensional subspaces with maximum discriminative power from high-dimensional subspaces in the supervised scenario. In particular, we make use of Riemannian geometry and optimization techniques on matrix manifolds to learn an orthogonal projection, which shows that the learning process can be formulated as an unconstrained optimization problem on a Grassmann manifold. With this natural geometry, any metric on the Grassmann manifold can theoretically be used in our model. Experimental evaluations on several data sets show that our approach results in significantly higher accuracy than other state-of-the-art algorithms.