Medical imaging sensors, such as positron emission tomography and single-photon emission computed tomography, can provide rich information, but each has its inherent drawbacks. In this scenario, multimodal sensor medical image fusion becomes an effective solution. The chief objective of medical imaging is to extract as much preponderant and complementary information as possible from the source into a single output that can play a critical role in medical diagnosis and clinical operations. In this paper, a novel fusion method is presented for multimodal sensor medical images, based on local difference (LD) in non-subsampled domain. In this method, the source medical images are first decomposed into low-frequency and high-frequency subimages, via non-subsampled schemes. Then, the coefficients of sub-bands are fused by an operator, called LD. The final fused image is reconstructed, via the inverse non-subsampled schemes, with all composite coefficients. The proposed fusion method was applied in several clinical studies, and the results show that it is a much more straightforward and effective method than some of the state-of-the-art methods, in terms of both subjective visual performance and objective evaluation results. Also, the performance of the proposed method was compared with that of two non-subsampled schemes, namely, non-subsampled contourlet transform and non-subsampled shearlet transform.
The emergence of multimodal medical imaging technology greatly increases the accuracy of clinical diagnosis and etiological analysis. Nevertheless, each medical imaging modal unavoidably has its own limitations, so the fusion of multimodal medical images may become an effective solution. In this paper, a novel fusion method on the multimodal medical images exploiting convolutional neural network (CNN) and extreme learning machine (ELM) is proposed. As a typical representative in deep learning, CNN has been gaining more and more popularity in the field of image processing. However, CNN often suffers from several drawbacks, such as high computational costs and intensive human interventions. To this end, the model of convolutional extreme learning machine (CELM) is constructed by incorporating ELM into the traditional CNN model. CELM serves as an important tool to extract and capture the features of the source images from a variety of different angles. The final fused image can be obtained by integrating the significant features together. Experimental results indicate that, the proposed method is not only helpful to enhance the accuracy of the lesion detection and localization, but also superior to the current state-of-the-art ones in terms of both subjective visual performance and objective criteria.
26 November 2010 in Science China Information Sciences
Presented is a novel technique to fuse multi‐sensor images using non‐subsampled shearlet transform (NSST) domain I 2 CM. First, multi‐scale and multi‐directional decompositions of source images can be performed by NSST. The classic ICM is improved to be I 2 CM which is then exploited to capture and extract the characteristics of each of the sub‐band images. Finally, the fused image is reconstructed from the fused sub‐images by adopting the inverse NSST. Experimental results demonstrate that the proposed image fusion scheme is very competitive compared with other conventional ones in both visual effect and objective evaluation criteria.
Despite the recent advancements in 3D object detection, the conventional 3D point cloud object detection algorithms have been found to exhibit limited accuracy for the detection of small objects. To address the challenge of poor detection of small-scale objects, this paper adopts the PointPillars algorithm as the baseline model and proposes a two-stage 3D target detection approach. As a cutting-edge solution, point cloud processing is performed using Transformer models. Additionally, a redefined attention mechanism is introduced to further enhance the detection capabilities of the algorithm. In the first stage, the algorithm uses PointPillars as the baseline model. The central concept of this algorithm is to transform the point cloud space into equal-sized columns. During the feature extraction stage, when the features from all cylinders are transformed into pseudo-images, the proposed algorithm incorporates attention mechanisms adapted from the Squeeze-and-Excitation (SE) method to emphasize and suppress feature information. Furthermore, the 2D convolution of the traditional backbone network is replaced by dynamic convolution. Concurrently, the addition of the attention mechanism further improves the feature representation ability of the network. In the second phase, the candidate frames generated in the first phase are refined using a Transformer-based approach. The proposed algorithm applies channel weighting in the decoder to enhance channel information, leading to improved detection accuracy and reduced false detections. The encoder constructs the initial point features from the candidate frames for encoding. Meanwhile, the decoder applies channel weighting to enhance the channel information, thereby improving the detection accuracy and reducing false detections. In the KITTI dataset, the experimental results verify the effectiveness of this method in small objects detection. Experimental results show that the proposed method significantly improves the detection capability of small objects compared with the baseline PointPillars. In concrete terms, in the moderate difficulty detection category, cars, pedestrians, and cyclists average precision (AP) values increased by 5.30%, 8.1%, and 10.6%, respectively. Moreover, the proposed method surpasses existing mainstream approaches in the cyclist category.
Accurately localizing and describing patients’ lesions has long been considered a crucial aspect of clinical diagnosis. The fusion of multimodal medical images provides a feasible solution to the above problem. Unfortunately, the trade-off between the fusion performance and heavy computation overhead remains a challenge. In this paper, a novel and effective fusion method for multimodal medical images is proposed. Firstly, framelet transform (FT) is introduced to decompose the source images into a series of low and high frequency sub-images. Next, we utilize the benefits of both steering kernel weighted guided filtering and side window filtering to successfully fuse sub-images. Finally, the inverse FT is employed to reconstruct the final fused image. To verify the effectiveness of the proposed fusion method, we fused several pairs of medical images covering different modalities in simulation experiments. The experimental results demonstrate that the proposed method yields better performance than current representative ones in terms of both visual quality and quantitative evaluation.