The detection and segmentation of key spacecraft components is a crucial prerequisite for the successful execution of on-orbit capture tasks. However, the existing spacecraft datasets are plagued by several problems. These include a lack of a unified dataset for component detection and segmentation, a lack of consideration of the motion states of the spacecraft, and a lack of consideration of extreme illuminations. These problems hinder the development of related research. In response to the above problems, this paper aims to bridge this gap by releasing a dataset for detecting and segmenting key spacecraft components. In contrast to synthetic images, the proposed dataset is built via hardware-in-the-loop capture. The dataset contains 2 types of spacecrafts, 9 types of key components, 3 types of illuminations (normal illumination, low illumination, and high saturation), and 2 types of motion states (approaching phase and hovering phase). Furthermore, we conduct extensive experiments on Spacecraft-DS with state-of-the-art methods for detection and segmentation tasks and provide comprehensive analysis. In addition, the dataset can also be used to evaluate many computer vision tasks, such as small object detection, etc. Our dataset and pre-trained models will be released at https://github.com/spacecraftds/Spacecraft-DS.

Dynamic targets in the environment can seriously affect the accuracy of simultaneous localization and mapping (SLAM) systems. This article proposes a novel dynamic visual SLAM method with inertial measurement unit (IMU) and deep learning for indoor dynamic blurred scenes, which improves the front end of ORB-SLAM2, combining deep learning with geometric constraint to make the dynamic feature points elimination more reasonable and robust. First, a multi-directional superposition blur augmentation algorithm is added to the YOLOv5s network to compensate for errors caused by fast-moving targets, camera shake and camera focus. Then, the fine-tuned YOLOv5s model is used to detect potential dynamic regions. Afterward, IMU measurements are introduced for rotation compensation to calculate the feature point velocity and estimate the motion speed of the camera, in order to estimate the real motion state of potential dynamic targets. Finally, real dynamic points will be removed and potential dynamic points will be reserved for subsequent pose estimation. Experiments are conducted on Technische Universität München dynamic dataset and in the real world. The results demonstrate that the proposed method has significant improvement compared with ORB-SLAM2, and has a more robust performance over some other state-of-the-art dynamic visual SLAM systems.

The extrinsic calibration of LiDAR-Camera system is a prerequisite for multimodal fusion. Target-based method can obtain precise extrinsic parameters offline. However, it is time-consuming and laborious, and during long-time application in real scenes, some unforeseen movement may cause LiDAR-Camera system to drift, which is necessary to be recalibrated. Therefore, this paper proposes a targetless calibration method, which first uses hand-eye estimator to reduce the cross-modality divergence for coarse calibration, and then introduces a Transformer-based deep network with local-to-global constraints to regress fine 6-DoF extrinsic parameters. The latter deep network is composed of four modules: Token Pyramid Module is used to reduce the size of Transformer; Transformer Fuser Module is designed for tightly coupling the point clouds and image data, exploiting its self-attention mechanism to fuse global spatial context of multimodality; Feature Injection Module injects local tokens of corresponding scale into global features for representation augmentation, in order to extract subsequent semantic features; Multi-constraint Module constructs a loss function including global regression constraint of 6-DoF extrinsic parameters, local depth projection and semantic edge constraint, further improving the robustness and generalization ability of the proposed network. Finally, an iterative refinement strategy is introduced to make the calibration results more precise. Experiments conducted on KITTI and Newer College dataset verify the proposed method has promising performance. Ablation studies also show the effectiveness of the proposed modules.

The existing Visual-inertial-LiDAR localization methods lack consideration of sensors degradation in challenging scene, such as underground space, where light condition is poor, text feature is scarce and geometric structure is degraded. To enhance the robustness of the multi-sensor fusion, this paper proposes a confidence factor based robust localization algorithm. It mainly consists of two parts. In front-end, a lightweight DCE-Net is used to improve the image quality under low illumination, and then an improved feature extraction method based on Line Segment Detector and Manhattan World (LSD-MW) is proposed to extract robust point-line features and construct Manhattan Frame (MF) structure constraint for subsequent optimization, making visual feature more reliable in underground space. In back-end, a novel confidence factor graph optimization strategy is proposed to enhance the robustness in the case of sensor degradation. Where adaptive confidence factors are designed to assess the reliability of LiDAR and camera features based on their feature matching degree. These confidence factors are leveraged to weight the sensors residual factors to construct an iterative function based on Graduated Nonconvexity (GNC), mitigating the influence of outliers on state estimation in degradation scene of underground space. Experiments conducted on public and real-world dataset collected by the UAV platform we built verify the proposed method has promising performance. Ablation studies also show the robustness of our method.

To solve the issue that LiDAR simultaneous localization and mapping (SLAM) suffers from geometric degenerate environment, several works have sought supplementary measurement information from camera instruments, while ignoring the temporal consistency of degeneration and timing of vision information introduction. In the case of a continuously moving platform, degeneration typically manifests as segments, which can lead to significant issues for SLAM performance. To tackle this, we propose a vision-assisted LiDAR-inertial SLAM based on multiple segments under a degenerate environment, which solves the degenerate problem on LiDAR SLAM in the way of segment-based optimization. The degenerate segments are identified using an eigenvalue-based degenerate detection method with a buffer mechanism. In addition, the visual re-projection residuals are enforced within degenerate segments, together with LiDAR odometry residuals and inertial odometry residuals, through a dual sliding window approach. Finally, considering the pose nodes within the degenerate segment as individual fixed nodes, a pose graph in the back end is applied to merge segments for final pose estimation. The experiments with an ablation study on both simulation and real-world data are implemented, and the result demonstrates the robustness of the proposed method, compared against state-of-the-art algorithms.

Aiming at the problems that current spatial layout mainly relies on manual experience, and there also exists long design period, low efficiency, and difficulty in realizing intelligence, this paper proposes a spatial layout method based on feature encoding and genetic-algorithm-optimized bidirectional long-term short-term memory neural network (GA-BiLSTM). First, a feature encoding algorithm is introduced to digitally extract the floorplan image information, and then the bidirectional structure is used based on LSTM to process the relationship between spatial sequences. Afterwards, a fine-tuned GA algorithm is proposed to optimize the number of hidden layers, the number of hidden and output layer neurons of the network to enhance the applicability and prediction accuracy of the neural network model. Finally, the optimization effect of the spatial layout scheme is realized. The experimental results indicates that the average prediction accuracy, precision, and recall of the proposed method on the test set are 94.65%, 94.53% and 94.31%, respectively, which are 10.08%, 10.69% and 11.42% higher than the LSTM model.