16 September 2020 in IEEE Transactions on Geoscience and Remote Sensing
Pan sharpening is used to fuse a low-resolution multispectral (MS) image and a high-resolution panchromatic (PAN) image to obtain a high-resolution MS image. This article proposes PSMD-Net, an end-to-end pan-sharpening method based on a multi-scale dense network. A shallow feature extraction layer (SFEL) extracts the shallow features from the original images, and these are used as an input to a global dense feature fusion (GDFF) network to learn the global features for image reconstruction. A multiscale dense block (MDB) is designed to fully extract the spatial and spectral information from the shallow features in the GDFF network. In the proposed network, multiple MDBs are stacked to extract rich, multi-scale dense hierarchical features, and a global dense connection (GDC) is designed to allow direct connections from the state of the current MDB to all subsequent MDBs to extract more advanced features. The extracted hierarchical features are sent to the global feature fusion layer (GFFL) to adaptively learn the global features for image reconstruction. Finally, global residual learning (GRL) is adopted to force the network to pay more attention to the changing part of the image. We perform experiments on simulated and real data from WorldView-2 and WorldView-3 satellites. Visual and quantitative assessment results demonstrate that PSMD-Net yields higher-resolution fusion images than the state-of-the-art methods.
1 January 2017 in Mathematical Problems in Engineering
Sparse representation has recently attracted enormous interests in the field of image super‐resolution. The sparsity‐based methods usually train a pair of global dictionaries. However, only a pair of global dictionaries cannot best sparsely represent different kinds of image patches, as it neglects two most important image features: edge and direction. In this paper, we propose to train two novel pairs of Direction and Edge dictionaries for super‐resolution. For single‐image super‐resolution, the training image patches are, respectively, divided into two clusters by two new templates representing direction and edge features. For each cluster, a pair of Direction and Edge dictionaries is learned. Sparse coding is combined with the Direction and Edge dictionaries to realize super‐resolution. The above single‐image super‐resolution can restore the faithful high‐frequency details, and the POCS is convenient for incorporating any kind of constraints or priors. Therefore, we combine the two methods to realize multiframe super‐resolution. Extensive experiments on image super‐resolution are carried out to validate the generality, effectiveness, and robustness of the proposed method. Experimental results demonstrate that our method can recover better edge structure and details.
7 August 2018 in Multimedia Tools and Applications
29 August 2024 in IEEE Transactions on Instrumentation and Measurement
Point cloud analysis has been significantly influenced by the PointNet family which has achieved remarkable accuracy across various benchmarks. However, enhancing the efficiency of these models without compromising performance remains a challenging task. In this paper, we investigate the efficiency of the state-of-the-art point-based method PointNeXt without degrading its performance and propose a more efficient and powerful network named Efficient PointNet (EP-Net). In particular, we propose a decoupled feature aggregation (DFA) module that decouples the learning of geometric and semantic features to capture more discriminative information, rather than the vanilla set abstraction (SA) module aggregating them for coarse joint processing. Besides, we develop a more effective sampling strategy for EP-Net, compressing processing complexity from O ( n log( n )) to linear. It selects points nearest to the center of each non-empty grid cell as the new sampling points. Impressively, we achieve a dual increase in both speed and accuracy, attributed to the more refined feature aggregation and more efficient sampling. Furthermore, we demonstrate the extensibility of our proposal towards other point cloud networks, from the perspective either of boost efficiency or performance prompt. Conditioned on extensive experiments on point cloud classification and segmentation tasks, EP-Net reports a state-of-the-art performance with 73.3% mean IoU on S3DIS Area5, boosting their baseline PointNeXt by 2.5%, with only 10.6% FLOPs and 37% parameters. On extensibility experiments, our strategy boosts PointMLP with an improvement of 0.9% mAcc on the ScanObjectNN dataset.
12 November 2020 in IEEE Access
Human pose estimation has attracted enormous interest in the field of human action recognition. When the human pose is complex (such as pose distortion, pose reversal, etc.) or there is background interference (multi-target, shadow, etc.), the keypoints obtained by existing methods of human pose estimation often have incorrect positioning, category, and connection. This paper proposes a novel human pose estimation network KACNet via the keypoint association constraints. The Channel-1 of KACNet is constrained by the distance loss function to obtain the position of keypoints, and the Channel-2 of KACNet is constrained by the association loss function to obtain the relationship of keypoints. Then, the position and relationship of keypoints are fused by the weighted loss function to obtain the keypoints with accurate location, classification, and connection. Experiments on a large number of public datasets and Internet data show that our method can effectively suppress background interference to improve the accuracy of complex human pose estimation. Compared with state-of-the-art human pose estimation methods, the proposed methods can accurately locate, classify, and connect the human body keypoints robustly.
15 October 2013 in Multimedia Tools and Applications