Given a set of calibrated images, Multiple View Stereo (MVS) applies end-to-end depth inference network to recover scene structure. However, previous methods designed pixel-visibility modules to aggregate cross-view cost, ignoring the consistency assumption of 2D contextual features in the 3D depth direction. The current multi-stage depth inference model also relies on intensive depth samples, which requires high memory consumption. To alleviate these problems, this work exploits edge-assisted epipolar Transformer for multi-view depth inference. The improvements of this work are summarized as follows: 1) The epipolar Transformer block is developed for reliable cross-view cost aggregation, and the edge detection branch is designed to constrain the consistency of epipolar geometry and edge features. 2) The dynamic depth range sampling mechanism based on probability volume is adopted to improve the accuracy of uncertain areas. Comprehensive comparisons with the state-of-the-art works indicate that our work can reconstruct dense scene representations with limited memory bottleblock Note to Practitioners —Learning-based MVS can obtain dense point clouds with accurate depth map estimation, which are widely applied in the fields of unmanned driving, battlefield environment perception and robot navigation. MVS-based scene reconstruction technology is the premise of the subsequent planning, decision-making and control of the human-machine system. To obtain dense scene representation with limited memory and runtime, this work proposes a multi-view stereo network with edge-assisted epipolar Transformer. Experiments on public benchmarks verify the feasibility and effectiveness of our model, which has good potential in battlefield environment reconstruction and human-computer interaction fields, and can provide intuitive and dense scene representation for decision-making assistance.

To study the relationship between surface electromyography (sEMG) and joint movement, and to provide reliable reference information for the exoskeleton control, the sEMG and the corresponding movement of the knee during the normal walking of adults have been measured. After processing the experimental data, the estimation model for knee movement from sEMG was established using the novel method of random forest with principal component analysis (RFPCA). The influence of the sample size and the previous sEMG data on the prediction efficiency was analyzed. The estimation model was not sensitive to the sample size when samples increased to a certain value, and the results of different previous sEMG showed that the prediction accuracy of the estimation models did not always improve with the increasing features of input. By comparing the estimation model of back propagation neural network with principal component analysis (BPPCA), it was found that RFPCA was suitable for all participants in the experiment with less execution time, and the root mean square error was around 5° which was lower than BPPCA with errors varying from 7° to 25°. Therefore, it was concluded that the RFPCA method for the estimation of knee movement from sEMG is feasible and could be used for motion analysis and the control of exoskeleton.

Multiple-View Stereo (MVS) aims to reconstruct the dense 3D representations of scenes. MVS has potential applications in the fields of autonomous driving (unstructured environment construction) and robotic navigation (visual-inertial navigation). To mitigate the error of depth estimation in low-textured or occluded regions, this work proposes a two-stage multi-view stereo network for fast and accurate depth estimation. The improvements of this work over the state of the art are as follows: 1) Sparse costs are constructed to jointly predict the initial depth map and surface normal by cost regularization, which proves that the surface normals can be estimated in this way with low memory consumption. 2) A new edge refinement block is developed to refine the coarse surface normal to obtain a fine-grained surface normal map. 3) Instead of using the general variance-based metric to equally aggregate cost, a new content-adaptive cost aggregation mechanism based on the similarity of the neighboring surface normal is designed for reliable cost aggregation. To the best of our knowledge, the proposed work is the first trainable network that leverages surface normal as guidance to capture neighboring pixel-visibility, which is an effective supplement to existing depth/normal estimation frameworks. Experimental results indicate that our method can not only achieve accurate depth estimation for scene perception but also make no concession to the real-time performance and limited memory bottleblock. Multiple-view stereo (MVS) aims to reconstruct the dense 3D representations of scenes. It is widely used in the fields of industrial measurement, autonomous driving, and robotic navigation. To mitigate the error of depth estimation in challenging scenarios, this work proposes a two-stage multi-view stereo network for fast and accurate depth estimation. Our method is the first trainable network that leverages surface normal as pixel-visibility guidance to aggregate reliable cost, which could achieve accurate depth estimation and provide the perception ability for the robot. The proposed method has great potential in the fields of 3D reconstruction, industrial measurement, and robotic navigation to estimate real-time and accurate depth with limited memory consumption.

The tracked-wheeled mobile robot has gained significant attention in military, agricultural, construction, and other fields due to its exceptional mobility and off-road capabilities. Therefore, it is an ideal choice for reconnaissance and exploration tasks. In this study, we proposed a multi-jointed tracked-wheeled compound mobile robot that can overcome various terrains and obstacles. Based on the characteristics of multi-jointed robots, we designed two locomotion modes for the robot to climb stairs and established the kinematics/dynamics equations for its land movement. We evaluated the robot’s stability during slope climbing, its static stability during stair climbing, and its ability to cross trenches. Based on our evaluation results, we determined the key conditions for the robot to overcome obstacles, the maximum height it can climb stairs, and the maximum width it can cross trenches. Additionally, we developed a simulation model to verify the robot’s performance in different terrains and the reliability of its stair-climbing gait. The simulation results demonstrate that our multi-jointed tracked-wheeled compound mobile robot exhibits excellent reliability and adaptability in complex terrain, indicating broad application prospects in various fields and space missions.

Multiple-view stereo has potential applications in robotic operations and autonomous driving (unstructured environment construction, visual servo). With assisted depth information, inertial navigation systems can achieve precise navigation. It is, especially suitable for GPS failures in complex environments. Accurate depth estimation is a challenge in low-textured or occluded regions. To alleviate the inference of incorrect depth, a multi-stage pixel-visibility learning-based stereo network is presented in this paper. Its improvements are as follows: 1) a new content-adaptive cost volume aggregation mechanism based on neighboring pixel-wise visibility is designed to effectively produce more accurate and smoother depth map predictions in the object boundary. 2) global convolution block and boundary refinement block are developed to regularize its cost volume, they can learn the inherent constraints of feature matching correspondence and effectively mitigate the depth estimation uncertainty in low-textured regions. 3) a new loss function is designed to measure the uncertainty of predicted probability distribution and enhance the reliability of depth map inference. Experimental results on the indoor DTU datasets and the outdoor Tanks & Temples datasets indicate that our method can achieve superior performance and has a powerful generalization ability, which is comparable to state-of-the-art works.