Given a set of calibrated images, Multiple View Stereo (MVS) applies end-to-end depth inference network to recover scene structure. However, previous methods designed pixel-visibility modules to aggregate cross-view cost, ignoring the consistency assumption of 2D contextual features in the 3D depth direction. The current multi-stage depth inference model also relies on intensive depth samples, which requires high memory consumption. To alleviate these problems, this work exploits edge-assisted epipolar Transformer for multi-view depth inference. The improvements of this work are summarized as follows: 1) The epipolar Transformer block is developed for reliable cross-view cost aggregation, and the edge detection branch is designed to constrain the consistency of epipolar geometry and edge features. 2) The dynamic depth range sampling mechanism based on probability volume is adopted to improve the accuracy of uncertain areas. Comprehensive comparisons with the state-of-the-art works indicate that our work can reconstruct dense scene representations with limited memory bottleblock Note to Practitioners —Learning-based MVS can obtain dense point clouds with accurate depth map estimation, which are widely applied in the fields of unmanned driving, battlefield environment perception and robot navigation. MVS-based scene reconstruction technology is the premise of the subsequent planning, decision-making and control of the human-machine system. To obtain dense scene representation with limited memory and runtime, this work proposes a multi-view stereo network with edge-assisted epipolar Transformer. Experiments on public benchmarks verify the feasibility and effectiveness of our model, which has good potential in battlefield environment reconstruction and human-computer interaction fields, and can provide intuitive and dense scene representation for decision-making assistance.

Learning-based stereo matching methods have produced remarkable results in recent years. However, typical supervised learning-based methods always suffer from the non-negligible problem of costly and time-consuming depth annotations. To mitigate this issue, in this work, a multi-stage unsupervised stereo matching method based on the cascaded Siamese network is proposed. To obtain a better performance on depth annotations, the improvements of this work are as follows. Firstly, sparse costs are constructed to predict the coarse disparity, and an adaptive sampling strategy is developed to dynamically adjust the sampling interval and effectively narrow the disparity search range. The proposed cost sparse and sampling strategy can certainly guarantee the accuracy of disparity estimation under the limited memory requirements. Then, geometric constraints with left and right semantic features are integrated into the loss function to learn the inherent matching correspondences. Next, information entropy of the probability volume is used to measure the quality of estimated disparity and designed as weighted guidance for the photometric loss. Finally, a pixel-wise disparity refinement module is designed to achieve high-resolution disparity estimation at the final stage. Experimental results on the datasets, including SceneFlow and KITTI, show the effectiveness and practicability of our method with limited memory consumption and running time.

Redundant robot manipulators play a significant role in modern industry. In this paper, we propose a solution scheme to the trajectory tracking problem of redundant robot manipulator with physical constraints through Zhang neural dynamics method. Such problem is integrated into a time-varying system consisting of time-varying nonlinear equation (TVNE) and time-varying linear inequality (TVLI) and solved online by the varying-parameter Zhang neural dynamics (VPZND) model. It is ensured that the redundant robot manipulator can still perform the tracking task perfectly under the coexistence of time-varying bounded noise and physical constraints. Theoretical analysis proves that this VPZND model also has an explicit fixed convergence time. Numerical experiments confirm the feasibility of our VPZND model for TVLI. The trajectory tracking problem of the redundant robot manipulator with six or three degrees of freedom under the dual influence of physical constraints and noise is perfectly solved by the VPZND model, which is enough to verify its practical value.

As a non-invasive brain imaging technology, functional magnetic resonance imaging (fMRI) provides a basic tool for brain functional network modeling and brain disease diagnosis. Problems, such as large number of parameters, low training efficiency, and poor interpretability, are encountered in mainstream models because of the high complexity of fMRI and brain networks. To solve these problems, a novel structure feature combined graph neural network (SFC-GNN) with a low number of parameters is proposed. In particular, SFC-GNN is composed of 1) the graph convolution layer of the brain region perception and 2) the node pooling layer of the graph structure feature (GSF). It also receives the sparse brain graph modeled by each subject’s fMRI as input. Especially, the GSF layer can select brain regions that are important for classification, thereby localizing all active regions related to brain disease. Moreover, a group network is constructed according to the correlation among subjects, and SFC-GNN can be extended further to a node classification model to achieve better diagnosis performance. The proposed method has been validated on the ABIDE and ADNI datasets, thereby showing the effectiveness of our proposed method in various experiments.

Given the precalibrated image pairs, stereo matching aims to infer the scene depth information in real-time, which has important research value in the fields of high-precision 3-D reconstruction of the Earth’s surface, automatic driving and unmanned aerial vehicle (UAV) navigation. The cost volume-based stereo matching method adopts a coarse-to-fine manner to construct cascaded cost volume, and applies 3-D convolution to capture the correspondence of feature matching to infer the disparity map, which achieves comparable performance. However, the existing method has difficulty dealing with jitter regions with disparity change, and direct disparity regression easily leads to overfitting of cost volume regularization. To alleviate the above two problems, this work proposes an end-to-end disparity estimation network based on Transformer. Its specific improvements are as follows. 1) The cross-view feature interaction module based on Transformer is introduced to realize the feature interaction of global context information. 2) A parallax attention mechanism is designed to impose global geometric constraints on the epipolar line to improve the reliability of feature matching. 3) Focal loss is applied for the training of the disparity classification model to emphasize one-hot supervision in ambiguous regions. Comprehensive experiments on public datasets Sceneflow, KITTI2015, ETH3D, and aerial WHU datasets validate that the proposed work can effectively enhance the performance of disparity estimation.

Multiple-View Stereo (MVS) aims to reconstruct the dense 3D representations of scenes. MVS has potential applications in the fields of autonomous driving (unstructured environment construction) and robotic navigation (visual-inertial navigation). To mitigate the error of depth estimation in low-textured or occluded regions, this work proposes a two-stage multi-view stereo network for fast and accurate depth estimation. The improvements of this work over the state of the art are as follows: 1) Sparse costs are constructed to jointly predict the initial depth map and surface normal by cost regularization, which proves that the surface normals can be estimated in this way with low memory consumption. 2) A new edge refinement block is developed to refine the coarse surface normal to obtain a fine-grained surface normal map. 3) Instead of using the general variance-based metric to equally aggregate cost, a new content-adaptive cost aggregation mechanism based on the similarity of the neighboring surface normal is designed for reliable cost aggregation. To the best of our knowledge, the proposed work is the first trainable network that leverages surface normal as guidance to capture neighboring pixel-visibility, which is an effective supplement to existing depth/normal estimation frameworks. Experimental results indicate that our method can not only achieve accurate depth estimation for scene perception but also make no concession to the real-time performance and limited memory bottleblock. Multiple-view stereo (MVS) aims to reconstruct the dense 3D representations of scenes. It is widely used in the fields of industrial measurement, autonomous driving, and robotic navigation. To mitigate the error of depth estimation in challenging scenarios, this work proposes a two-stage multi-view stereo network for fast and accurate depth estimation. Our method is the first trainable network that leverages surface normal as pixel-visibility guidance to aggregate reliable cost, which could achieve accurate depth estimation and provide the perception ability for the robot. The proposed method has great potential in the fields of 3D reconstruction, industrial measurement, and robotic navigation to estimate real-time and accurate depth with limited memory consumption.