The fusion of multimodal medical images, particularly CT and MRI, is driven by the need to enhance the diagnostic process by providing clinicians with a single, comprehensive image that encapsulates all necessary details. Existing fusion methods often exhibit a bias towards features from one of the source images, making it challenging to simultaneously preserve both structural information and textural details. Designing an effective fusion method that can preserve more discriminative information is therefore crucial. In this work, we propose a Coupled Feature-Learning GAN (CFGAN) to fuse the multimodal medical images into a single informative image. The proposed method establishes an adversarial game between the discriminators and a couple of generators. First, the coupled generators are trained to generate two real-like fused images, which are then used to deceive the two coupled discriminators. Subsequently, the two discriminators are devised to minimize the structural distance to ensure the abundant information in the original source images is well-maintained in the fused image. We further empower the generators to be robust under various scales by constructing a discriminative feature extraction (DFE) block with different dilation rates. Moreover, we introduce a cross-dimension interaction attention (CIA) block to refine the feature representations. The qualitative and quantitative experiments on common benchmarks demonstrate the competitive performance of the CFGAN compared to other state-of-the-art methods.
29 November 2024 in IEEE Wireless Communications Letters
To reduce the required pilot and feedback overhead in reconfigurable intelligent surface (RIS)-aided wireless communication systems under frequency division duplex (FDD) mode, a deep distributed learning-based joint channel estimation and feedback framework is proposed. Specifically, in the channel estimation stage, an RIS channel estimation network (RIS-CENet) is designed to reconstruct the channel from limited pilots by using the dual-channel attention mechanism. In the channel feedback stage, the RIS channel feedback network (RIS-CFNet), based on a multi-head attention mechanism, is designed to capture the global features of high-dimensional channel matrices, thereby enabling their effective compression. Furthermore, an efficient distributed learning framework is proposed to achieve channel estimation and feedback tasks in complex and rapidly changing wireless channel environments. Numerical results show that the performance of the proposed distributed learning architecture is comparable to that of the centralized learning architecture. Distributed scheme I has better feedback accuracy, while distributed scheme II can effectively reduce the complexity of the system.
29 June 2022 in Journal of Circuits, Systems and Computers
Image compression approaches using implicit neural representation (INR) have recently gained attention for their lightweight nature, compactness, and fast decoding, showing promise for edge computing in consumer devices. Specifically, INR-based image compression methods implicitly store each image within a lightweight neural network, which serves as a compact representation of the image. However, most existing methods are limited to representing single-quality images with fixed-size models, which necessitates training separate models independently for images at varying quality levels, leading to additional training and storage costs. To tackle this problem, we propose a progressive image compression method based on Width-Depth Scalable Implicit Neural Representation (WDSINR), which are composed of executable sub-networks of varying scales. By adjusting the scale of the sub-networks, WDS-INR can represent images at different quality levels while supporting progressive transmission. The scalable architecture of WDSINR makes it well-suited for deployment on mobile and IoTs devices. Furthermore, we propose a band-limited initialization scheme that enhances both the representation capabilities and training stability of the WDS-INR. Finally, we introduce a meta-learning approach to the base sub-network to accelerate encoding (4 × faster). Experimental results demonstrate that the proposed method outperforms the baseline in rate-distortion performance (+0.28 dB PSNR), while enabling scalable bit-rates with progressive decoding.
28 January 2025 in IEEE Transactions on Multimedia
The learned image compression (LIC) technique has surpassed the state-of-the-art traditional codecs (H.266/VVC) in case of rate-distortion (R-D) performance. Its real-time deployments are far advanced. In order to achieve more flexible deployments, an LIC technique should be flexible in adjusting its computational complexity and rate as demanded by a situation and its environment. In this paper, we propose a unified Rate-Distortion-Complexity (R-D-C) framework for LIC under channel energy concentration criteria. Specifically, we first introduce an Energy Asymptotic Nonlinear Transformation (EANT) designed to directly concentrate on the channel energy of latent representations, thus laying the groundwork for a scalable entropy coding. Next, leveraging this energy concentration characteristic, we propose a corresponding Heterogeneous Scalable Entropy Model (HSEM) for flexibly scaling bitstreams as needed. Finally, utilizing the proposed EANT, we construct a fine-grained scalable codec for formulating, in combination with HSEM, a comprehensive scalable R-D-C framework under the energy concentration criteria. The obtained experimental results demonstrate that the proposed method could enable seamless transitions between 13 different widths of sub-models within a single network, allowing for fine-grained control over the model bitrate, complexity, and hardware inference time. Additionally, the proposed method exhibits competitive R-D performance compared to many existing methods.