23 September 2024 in IEEE Internet of Things Journal
This article introduces a novel method for transmitting video data over noisy wireless channels with high efficiency and controllability. The method derivates from model division multiple access (MDMA) to extract common semantic features from video frames. It also uses deep joint source-channel coding (JSCC) as the main framework to establish communication links and deal with channel noise. An entropy-based semantic importance coding scheme is developed to adjust the data amount accurately and explicitly. We name our method as model division video semantic communication (MDVSC). The main steps of our approach are as follows: first, video frames are transformed into a latent space to reduce computational complexity and redistribute data. Then, common features and individual features are extracted, and semantic importance coding is applied to further eliminate redundant semantic information under the communication bandwidth constraint. We evaluate our method on standard video test sequences and compare it with traditional wireless video coding methods. The results show that MDVSC generally surpasses the conventional methods in terms of quality metrics and has the capability to control code length precisely. Moreover, additional experiments and ablation studies are conducted to demonstrate its potential for various tasks.
12 September 2024 in IEEE Transactions on Vehicular Technology
Point cloud, as a 3D representation, finds wide applications in domains such as autonomous driving, virtual reality (VR), and augmented reality (AR). However, traditional communication systems are not well-suited for large-scale point cloud data transmissions, as such systems operate at the bit level without leveraging the inherent semantic information of the point cloud. This paper introduces a point cloud-based semantic communication system (PCSC) that leverages AI-based encoding techniques to extract semantic information from the point cloud. Furthermore, joint source-channel coding (JSCC) technology is employed to overcome noise channel distortion and address the “cliff effect” prevalent in traditional communication methods. Additionally, the proposed system achieves controllable coding rates without requiring extensive network fine-tuning. By analyzing the significance of the encoded semantic vector, the method discards semantically-unimportant information, enhancing transmission efficiency. Moreover, the PCSC is integrated with the recently proposed model division multiple access (MDMA) technology, resulting in a multi-user point cloud MDMA transmission system (M-PCSC). Experimental results demonstrate that the proposed method surpasses traditional approaches by 10 dB under the PSNR D1 and PSNR D2 metrics while effectively mitigating the “cliff effect” in transmission Index Terms
5 May 2025 in IEEE Transactions on Communications
In this paper, we propose a novel semantic digital analog converter (sDAC) for the compatibility between semantic and digital communications. Most of the current semantic communication systems rely primarily on analog modulation, limiting their integration with digital communication systems, which are more common in practice. In fact, traditional quantization methods are unsuitable for semantic communication because they do not account for semantic information within symbols. These factors block the wide application of the semantic communication. To address these challenges, sDAC is proposed. It is a simple yet efficient and generative module used to realize digital and analog bi-directional conversion. The entire process is independent of any specific semantic model, modulation methods, or channel conditions. In the experiment section, the performance of sDAC is tested across different semantic models, semantic tasks, modulation methods, channel conditions and quantization orders. Test results show that the proposed sDAC has great generative properties and channel robustness.
16 January 2025 in IEEE Internet of Things Journal
As a novel communication paradigm, semantic communication (SemCom) can greatly improve communication efficiency, which has aroused extensive research by scholars worldwide. As one of the important aspects of digital communication nowadays, how to combine channel estimation with SemCom is an important research direction. In this article, based on orthogonal frequency-division multiplexing (OFDM) communication architecture, the semantic importance-aware reordering-enhanced SemCom system (SIARE-SC) is proposed, which utilizes the inequality of semantic symbols combined with channel estimation in OFDM systems to reduce the distortion caused by channel estimation interpolation error (CEIE) and further improve the signal recovery quality. To enhance the generalizability of the system, we extend the verification of the effectiveness of SIARE-SC in various scenarios with different sources, channels, and pilot patterns. Furthermore, the importance reordering method proposed in the SIARE-SC has good applicability and effectiveness, which can be used to be compatible with other SemCom systems and has a significant suppression effect on the peak-to-average power ratio (PAPR). Meanwhile, CEIE has been considered for the first time to be included in the analysis of SemCom distortion, and mathematically derive the performance expressions of SIARE-SC under different channel and pilot pattern scenarios from three perspectives, namely, channel bandwidth ratio (CBR), signal-to-noise ratio (SNR), and CEIE, to obtain the corresponding bound of performance. The proposed SIARE-SC is shown to significantly improve semantic performance in various scenarios by conducting a large number of experimental tests.
11 January 2024 in IEEE Internet of Things Journal
Nowadays, the need for high-quality image reconstruction and restoration is more and more urgent. However, most image transmission systems may suffer from image quality degradation or transmission interruption in the face of interference such as channel noise and link fading. To solve this problem, the a relay communication network for semantic image transmission based on shared feature extraction and hyperprior entropy compression (HEC) is proposed, where the shared feature extraction technology based on Pearson correlation is proposed to eliminate partial shared feature of extracted semantic latent feature. In addition, the HEC technology is used to resist the effect of channel noise and link fading and carried out, respectively, at the source node and the relay node. Experimental results demonstrate that compared with other recent research methods, the proposed system has lower transmission overhead and higher semantic image transmission performance. Particularly, compared with the element-distance-based scheme, the proposed shared feature extraction method based on Pearson correlation not only exhibits an approximate 0.2 advantage in multiscale structural similarity (MS-SSIM) but also efficiently eliminates the burden caused by the transmission of shared feature indexes.
16 December 2024 in IEEE Internet of Things Journal
With the rapid development of Internet of Things (IoT) technology, more sensors are required to operate in complex channel scenarios and under limited communication resources. Semantic communication, as an emerging paradigm, extracts, transmits, and reconstructs information at the semantic level, offering advantages, such as high compression rates and strong noise resistance. These features are expected to find widespread application across various IoT scenarios. However, widely used image similarity evaluation metrics like peak signal-to-noise ratio and multiscale structural similarity index primarily focus on pixel or structural features, making it challenging to accurately measure the loss of semantic-level information during transmission. This limitation poses challenges for the performance evaluation of visual semantic communication systems and restricts the emergence of more novel and efficient systems. To address this issue, we propose a new semantic evaluation metric-semantic similarity score (SeSS). This metric is based on Scene Graph Generation and graph matching techniques, transforming image similarity scores into graph matching scores. By manually annotating thousands of image pairs, we fine-tuned the hyperparameters within SeSS to align it more closely with human semantic perception. The performance of SeSS has been tested across various image datasets and specific IoT visual tasks. Experimental results demonstrate the effectiveness of SeSS in measuring differences in semantic-level information between images, making it a valuable tool for evaluating visual semantic communication systems. This development is expected to encourage the emergence of more robust systems suited for diverse IoT scenarios. The code of SeSS is openly available on https://github.com/FSR3340/Semantic_Similarty_ScoreGitHub.