Chang’e-5, China’s first unmanned lunar sample-return mission, was successfully landed in Northern Oceanus Procellarum on 1 December 2020. Determining the lander location precisely and timely is critical for both engineering operations and subsequent scientific research. Localization of the lander was performed using radio-tracking and image-based methods. The lander location was determined to be (51.92°W, 43.06°N) by both methods. Other localization results were compared for cross-validation. The localization results greatly contributed to the planning of the ascender lifting off from the lander and subsequent maneuvers, and they will contribute to scientific analysis of the returned samples and in situ acquired data.

In planetary rover missions, rover path planning is critical to ensure the safety and efficiency of the rover traverse and in situ explorations. In the Chang’e-4 (CE-4) mission, we have proposed and developed vision-based decision support methods comprising obstacle map generation and path evaluation for the rover. At each waypoint along the rover traverse, digital elevation model (DEM) is automatically generated, which is then used for obstacle map generation and path searching. For path evaluation, the searched path and the predicted wheel trajectories are projected onto the original images captured by different cameras to coincide with the real observation scenario. The proposed methods have been applied in the CE-4 mission to support teleoperation of the rover, and examples of multiple applications used in the mission are presented in this paper. Under the support of the proposed techniques, by the end of the 14th lunar day (3 February 2020), the rover has already travelled 367.25 meters on the far side of the Moon.

Topographic products are important for mission operations and scientific research in lunar exploration. In a lunar rover mission, high-resolution digital elevation models are typically generated at waypoints by photogrammetry methods based on rover stereo images acquired by stereo cameras. In case stereo images are not available, the stereo-photogrammetric method will not be applicable. Alternatively, photometric stereo method can recover topographic information with pixel-level resolution from three or more images, which are acquired by one camera under the same viewing geometry with different illumination conditions. In this research, we extend the concept of photometric stereo to photogrammetric-photometric stereo by incorporating collinearity equations into imaging irradiance model. The proposed photogrammetric-photometric stereo algorithm for surface construction involves three steps. First, the terrain normal vector in object space is derived from collinearity equations, and image irradiance equation for close-range topographic mapping is determined. Second, based on image irradiance equations of multiple images, the height gradients in image space can be solved. Finally, the height map is reconstructed through global least-squares surface reconstruction with spectral regularization. Experiments were carried out using simulated lunar rover images and actual lunar rover images acquired by Yutu-2 rover of Chang’e-4 mission. The results indicate that the proposed method achieves high-resolution and high-precision surface reconstruction, and outperforms the traditional photometric stereo methods. The proposed method is valuable for ground-based lunar surface reconstruction and can be applicable to surface reconstruction of Earth and other planets.

Visual odometry provides astronauts with accurate knowledge of their position and orientation. Wearable astronaut navigation systems should be simple and compact. Therefore, monocular vision methods are preferred over stereo vision systems, commonly used in mobile robots. However, the projective nature of monocular visual odometry causes a scale ambiguity problem. In this paper, we focus on the integration of a monocular camera with a laser distance meter to solve this problem. The most remarkable advantage of the system is its ability to recover a global trajectory for monocular image sequences by incorporating direct distance measurements. First, we propose a robust and easy-to-use extrinsic calibration method between camera and laser distance meter. Second, we present a navigation scheme that fuses distance measurements with monocular sequences to correct the scale drift. In particular, we explain in detail how to match the projection of the invisible laser pointer on other frames. Our proposed integration architecture is examined using a live dataset collected in a simulated lunar surface environment. The experimental results demonstrate the feasibility and effectiveness of the proposed method.

A light field camera can capture both radiance and angular information, providing a novel solution for depth estimation. The paper proposes two improved methods including distortion model optimization and depth estimation refinement for a lenslet light field camera. For distortion model optimization, a novel 14-parameter distortion model that involves sub-aperture images generation is applied to correct the light field camera images. For depth estimation refinement, an algorithm reducing the high influence of outliers on depth estimation in weak texture regions is proposed based on multi-view stereo matching using the cost volume. Experimental results show the projection error has decreased by about 30% and the depth root-mean-squared error on real world images has decreased by about 42% with our distortion correction method and depth estimation method compared with state of art algorithms. It verifies the correctness and effectiveness of our proposed methods and show significant improvement on accuracy of depth map estimation.