In recent years, image segmentation techniques based on deep learning have achieved many applications in remote sensing, medical, and autonomous driving fields. In space exploration, the segmentation of spacecraft objects by monocular images can support space station on-orbit assembly tasks and space target position and attitude estimation tasks, which has essential research value and broad application prospects. However, there is no segmentation network designed for spacecraft targets. This paper proposes an end-to-end spacecraft image segmentation network using the semantic segmentation network DeepLabv3+ as the basic framework. We develop a multi-scale neural network based on sparse convolution. First, the feature extraction capability is improved by the dilated convolutional network. Second, we introduce the channel attention mechanism into the network to recalibrate the feature responses. Finally, we design a parallel atrous spatial pyramid pooling (ASPP) structure that enhances the contextual information of the network. To verify the effectiveness of the method, we built a spacecraft segmentation dataset on which we conduct experiments on the segmentation algorithm. The experimental results show that the encoder+ attention+ decoder structure proposed in this paper, which focuses on high-level and low-level features, can obtain clear and complete masks of spacecraft targets with high segmentation accuracy. Compared with DeepLabv3+, our method is a significant improvement. We also conduct an ablation study to research the effectiveness of our network framework.

In this paper, we propose a novel 3D object detector KDA3D, which achieves high-precision and robust classification, segmentation, and localization with the help of key-point densification and multi-attention guidance. The proposed end-to-end neural network architecture takes LIDAR point clouds as the main inputs that can be optionally complemented by RGB images. It consists of three parts: part-1 segments 3D foreground points and generates reliable proposals; part-2 (optional) enhances point cloud density and reconstructs the more compact full-point feature map; part-3 refines 3D bounding boxes and adds semantic segmentation as extra supervision. Our designed lightweight point-wise and channel-wise attention modules can adaptively strengthen the “skeleton” and “distinctiveness” point-features to help feature learning networks capture more representative or finer patterns. The proposed key-point densification component can generate pseudo-point clouds containing target information from monocular images through the distance preference strategy and K-means clustering so as to balance the density distribution and enrich sparse features. Extensive experiments on the KITTI and nuScenes 3D object detection benchmarks show that our KDA3D produces state-of-the-art results while running in near real-time with a low memory footprint.

We present MCF3D, a multi-stage complementary fusion three-dimensional (3D) object detection network for autonomous driving, robot navigation, and virtual reality. This is an end-to-end learnable architecture, which takes both LIDAR point clouds and RGB images as inputs and utilizes a 3D region proposal subnet and second stage detector(s) subnet to achieve high-precision oriented 3D bounding box prediction. To fully exploit the strength of multimodal information, we design a series of fine and targeted fusion methods based on the attention mechanism and prior knowledge, including “pre-fusion”, “anchor-fusion”, and “proposal-fusion.” Our proposed RGB-Intensity form encodes the reflection intensity onto the input image to strengthen the representational power. Our designed proposal-element attention module allows the network to be guided to focus more on efficient and critical information with negligible overheads. In addition, we propose a cascade-enhanced detector for small classes, which is more selective against close false positives. Experiments on the challenging KITTI benchmark show that our MCF3D method produces state-of-the-art results while running in near real-time with a low memory footprint.

Remote sensing images usually contain abundant targets and complex information distributions. Consequently, networks are required to model both global and local information in the super-resolution (SR) reconstruction of remote sensing images. The existing SR reconstruction algorithms generally focus on only local or global features, neglecting effective feedback for reconstruction errors. Therefore, a Global Residual Multi-attention Fusion Back-projection Network (SRBPSwin) is introduced by combining the back-projection mechanism with the Swin Transformer. We incorporate a concatenated Channel and Spatial Attention Block (CSAB) into the Swin Transformer Block (STB) to design a Multi-attention Hybrid Swin Transformer Block (MAHSTB). SRBPSwin develops dense back-projection units to provide bidirectional feedback for reconstruction errors, enhancing the network’s feature extraction capabilities and improving reconstruction performance. SRBPSwin consists of the following four main stages: shallow feature extraction, shallow feature refinement, dense back projection, and image reconstruction. Firstly, for the input low-resolution (LR) image, shallow features are extracted and refined through the shallow feature extraction and shallow feature refinement stages. Secondly, multiple up-projection and down-projection units are designed to alternately process features between high-resolution (HR) and LR spaces, obtaining more accurate and detailed feature representations. Finally, global residual connections are utilized to transfer shallow features during the image reconstruction stage. We propose a perceptual loss function based on the Swin Transformer to enhance the detail of the reconstructed image. Extensive experiments demonstrate the significant reconstruction advantages of SRBPSwin in quantitative evaluation and visual quality.

This paper aims to apply real-time light-weight high-precision 3D detection for autonomous driving. We propose LIDAR-based 3D object detection based on foreground segmentation using a fully sparse convolutional network (FS ² 3D). We design a sparse convolutional backbone network and a sparse convolutional detection head to efficiently use the computing and memory resources and accelerate the inference. Instead of using the anchor-based method, we convert the detection problem into a foreground segmentation problem on a bird's-eye view. The sparse convolutional detection head predicts the objectness and bounding box on each active point on the sparse feature map. We design a new oriented bounding box coding method and corresponding loss functions. We predict the endpoints of two mutually perpendicular lines that pass through the foreground active points and indirectly predict the objects' oriented bounding box from these four endpoints. We use the indirectly calculated object center, size, and orientation as inputs of loss functions in the training step. Experiments on the KITTI dataset show that the sparse backbone network we designed is 2.2 times faster and 18.4 times fewer FLOPs than the dense backbone network. The average improvement of the loss functions based on the bounding box code is 1.1% and 0.8% on the BEV and 3D detection, respectively, compared to no addition of these losses. Moreover, FS ² 3D outperforms the state-of-the-art LIDAR-based method in speed and precision for both cars and cyclists.

The learning of discriminative features is the key for fine-grained image recognition. To better extract effective features and improve the accuracy of fine-grained image recognition, we propose a two-level progressive attention convolutional network (TPA-CNN) for fine-grained image recognition. The model includes a multi-channel attention-fusion (MCAF) module and a cross-layer element-attention (CEA) module. The MCAF module is used to find distinctive feature map channels which significantly responds to specific regions. Inspired by Hierarchical Bilinear Pooling model, The CEA module is further assign weight values to feature map elements. From the perspective of attention visualization map, MCAF module can focused on one or more positive regions, CEA module further locates the most discriminative regions by interaction between the feature map elements. The model can dynamically search the discriminative region of the image, not limited to the boost or crop a selected region. Compared to previous models basing on attention mechanism, the model can extract non-correlated part features which spread over object foreground areas, further improving the recognition accuracy. Experimental results on CUB-200-2011, FGVC-Aircraft, and Stanford Cars datasets demonstrate that the proposed TPA-CNN achieves competitive performance.