In this paper, we propose a class of high-efficiency deep joint source-channel coding methods that can closely adapt to the source distribution under the nonlinear transform, it can be collected under the name nonlinear transform source-channel coding (NTSCC). In the considered model, the transmitter first learns a nonlinear analysis transform to map the source data into latent space, then transmits the latent representation to the receiver via deep joint source-channel coding. Our model incorporates the nonlinear transform as a strong prior to effectively extract the source semantic features and provide side information for source-channel coding. Unlike existing conventional deep joint source-channel coding methods, the proposed NTSCC essentially learns both the source latent representation and an entropy model as the prior on the latent representation. Accordingly, novel adaptive rate transmission and hyperprior-aided codec refinement mechanisms are developed to upgrade deep joint source-channel coding. The whole system design is formulated as an optimization problem whose goal is to minimize the end-to-end transmission rate-distortion performance under established perceptual quality metrics. Across test image sources with various resolutions, we find that the proposed NTSCC transmission method generally outperforms both the analog transmission using the standard deep joint source-channel coding and the classical separation-based digital transmission. Notably, the proposed NTSCC method can potentially support future semantic communications due to its content-aware ability and perceptual optimization goal.
16 November 2022 in IEEE Journal on Selected Areas in Communications
In this paper, we design a new class of high-efficiency deep joint source-channel coding methods to achieve end-to-end video transmission over wireless channels. The proposed methods exploit nonlinear transform and conditional coding architecture to adaptively extract semantic features across video frames, and transmit semantic feature domain representations over wireless channels via deep joint source-channel coding. Our framework is collected under the name deep video semantic transmission (DVST). In particular, benefiting from the strong temporal prior provided by the feature domain context, the learned nonlinear transform function becomes temporally adaptive, resulting in a richer and more accurate entropy model guiding the transmission of current frame. Accordingly, a novel rate adaptive transmission mechanism is developed to customize deep joint source-channel coding for video sources. It learns to allocate the limited channel bandwidth within and among video frames to maximize the overall transmission performance. The whole DVST design is formulated as an optimization problem whose goal is to minimize the end-to-end transmission rate-distortion performance under perceptual quality metrics or machine vision task performance metrics. Across standard video source test sequences and various communication scenarios, experiments show that our DVST can generally surpass traditional wireless video coded transmission schemes. The proposed DVST framework can well support future semantic communications due to its video content-aware and machine vision task integration abilities.
8 July 2024 in IEEE Transactions on Cognitive Communications and Networking
As one of the key techniques to realize semantic communications, end-to-end optimized neural joint source-channel coding (JSCC) has made great progress over the past few years. A general trend in many recent works pushing the model adaptability or the application diversity of neural JSCC is based on the convolutional neural network (CNN) backbone, whose model capacity is yet limited, inherently leading to inferior system coding gain against traditional coded transmission systems. In this paper, we establish a new neural JSCC backbone that can also adapt flexibly to diverse channel conditions and transmission rates within a single model, our open-source project aims to promote the research in this field. Specifically, we show that with elaborate design, neural JSCC codec built on the emerging Swin Transformer backbone achieves superior performance than conventional neural JSCC codecs built upon CNN, while also requiring lower end-to-end processing latency. Paired with two spatial modulation modules that scale latent representations based on the channel state information and target transmission rate, our baseline SwinJSCC can further upgrade to a versatile version, which increases its capability to adapt to diverse channel conditions and rate configurations. Extensive experimental results show that our SwinJSCC achieves better or comparable performance versus the state-of-the-art engineered BPG + 5G LDPC coded transmission system with much faster end-to-end coding speed, especially for high-resolution images, in which case traditional CNN-based JSCC yet falls behind due to its limited model capacity.
Thanks to the inherent characteristics of flexible mobility and autonomous operation, unmanned aerial vehicles (UAVs) will inevitably be integrated into 5G/B5G cellular networks to assist remote sensing for real-time assessment and monitoring applications. Most existing UAV-assisted data collection schemes focus on optimizing energy consumption and data collection throughput, which overlook the temporal value of collected data. In this article, we employ Age of Information (AoI) as a performance metric to quantify the temporal correlation among data packets consecutively sampled by the Internet of Things (IoT) devices, and investigate an AoI-energy-aware data collection scheme for UAV-assisted IoT networks. We aim to minimize the weighted sum of expected average AoI, propulsion energy of UAV, and the transmission energy at IoT devices, by jointly optimizing the UAV flight speed, hovering locations, and bandwidth allocation for data collection. Considering the system dynamics, the optimization problem is modeled as a Markov decision process. To cope with the multidimensional action space, we develop a twin-delayed deep deterministic (TD3) policy gradient-based UAV trajectory planning algorithm (TD3-AUTP) by introducing the deep neural network (DNN) for feature extraction. Through simulation results, we demonstrate that our proposed scheme outperforms the deep $Q$ -network and actor–critic-based algorithms in terms of achievable AoI and energy efficiency.
13 August 2024 in IEEE Communications Surveys & Tutorials
Information and communication technology (ICT) has been an essential part of modern society. However, the current communication systems are not sufficient to meet the demands of emerging applications. Intellicise (intelligent and concise) wireless networks, with their inherent characteristics of intelligence-endogenous and primitive-concise, have been proposed as a promising research direction. In this paper, we focus on intellicise wireless networks from semantic communication (SemCom). We present a comprehensive framework of intellicise wireless networks, including components such as brain for intellicise wireless networks (BIWN), intellicise signal processing, intellicise information transmission, intellicise network organization, and intellicise service bearing. We also investigate the enabling technologies and driving factors of intellicise wireless networks. Subsequently, we introduce the applications of intellicise wireless networks and envision new services. Finally, we outline the challenges of implementing intellicise wireless networks from a broad perspective and discuss possible solutions.
25 January 2022 in IEEE Internet of Things Journal
Federated learning (FL) through its novel applications and services has enhanced its presence as a promising tool in the Internet of Things (IoT) domain. Specifically, in a multiaccess edge computing setup with a host of IoT devices, FL is most suitable since it leverages distributed client data to train high-performance deep learning (DL) models while keeping the data private. However, the underlying deep neural networks (DNNs) are huge, preventing its direct deployment onto resource-constrained computing and memory-limited IoT devices. Besides, frequent exchange of model updates between the central server and clients in FL could result in a communication bottleneck. To address these challenges, in this article, we introduce GWEP, a model compression-based FL method. It utilizes joint quantization and model pruning to reap the benefits of DNNs while meeting the capabilities of resource-constrained devices. Consequently, by reducing the computational, memory, and network footprint of FL, the low-end IoT devices may be able to participate in the FL process. In addition, we provide theoretical guarantees of FL convergence. Through empirical evaluations, we demonstrate that our approach significantly outperforms the baseline algorithms by being up to 10.23 times faster with 11 times lesser communication rounds, while achieving high-model compression, energy efficiency, and learning performance.