22 February 2023 in IEEE Transactions on Communications
A high-performance communication receiver desires a sufficient and accurate recognition of transmitted sources. In this paper, we propose a sequential decoding algorithm for the robust reception of sources with natural redundancy (NR) over the AWGN channel. To fully exploit the abundant NR in the exemplified English text sources, a causal language modeling (CLM) with a powerful Transformer decoder neural network (NN) structure is adopted for modeling the joint probability distribution. For versatility of byte-level tokenization, the UTF-8 encoding scheme is considered for the Wikipedia corpus, namely the English edition of the Wiki-40B dataset. The tree search-based M-algorithm (MA) is integrated with CLM, denoted as CLM-MA algorithm, to synthesize the a priori probability of information sequence and likelihood values of polluted symbols. Both simulation experiments and hardware platform evaluations are conducted to analyze the performance of the CLM-MA algorithm. With an adequately large M value, tremendous performance profit is achievable, especially for high-efficiency modulation levels and extremely noisy conditions. This mechanism of adopting the pre-trained NN model merely desires self-supervised learning architecture and eliminates the requirement of high-cost and accurate labels, which paves a new way for communication receiver design.
30 November 2022 in IEEE Transactions on Cognitive Communications and Networking
High-speed information transmission system demands more powerful communication receivers. In this paper, we integrate deep learning algorithms into constructing waveform level receiver, which is capable of harnessing natural redundancy (NR) in raw message sources. Owing to end-to-end learning paradigm of neural network (NN), the proposed NN receiver can accomplish joint symbol timing synchronization and demodulation based on oversampled waveform sequences directly. The prevalent bidirectional LSTM (BiLSTM) and Transformer structures, suitable for excavating sequential correlation of complex sources and waveform sequences, are considered as the backbone of NN receivers. Under the cases of clean label and noisy label, we conceive two novel training algorithms and interpret their principles in theory. The clean label bits are accessible from original source encoding schemes, while noisy label bits are available from the decision results of canonical receivers with independently noised waveform sequences. When noisy labels possess sufficiently small error rates, optimized NN receiver is competitive with that adopting the accurate labels. As for Markov sources and natural language text sources, simulation results show that the proposed NN receivers’ performances are superior to the classical receivers without a priori knowledge, especially when high spectral efficiency modulation schemes are encountered and random noise is significant.
25 July 2022 in IEEE Communications Letters
Pre-trained Models (PTMs) have reached the state-of-the-art (SOTA) on many Natural Language Processing (NLP) tasks and Computer Vision (CV) tasks, and are called the foundation models of artificial intelligence (AI) systems. In this paper, we introduce PTMs into communication receiver design, which is the first attempt to apply PTMs to the communications field. The powerful capability of PTMs to capture knowledge from massive data enables the PTMs-based receivers to utilize the natural redundancy (NR) that widely exists in the transmitted sources to obtain high performance gains. Intended for uncompressed Chinese text sources, we design a high-performance receiver based on the Bidirectional Encoder Representation from Transformers (BERT) model in the NLP field. We achieve an approximate maximum of posterior marginals (MPM) estimation by obtaining the conditional distribution of the input data from the pre-trained BERT model, and the performance of the approximate MPM estimation can be improved by fine-tuning the BERT model with self-supervised learning. Simulation results demonstrate that the performance of our designed PTMs-based receiver significantly outperforms the classical receiver.
20 December 2021 in IEEE Communications Letters
Power of deep neural networks (NNs) has enabled tremendous effective applications for communication receiver design. In this letter, we demonstrate the possibility of constructing merely one NN to straightforwardly recover bit messages from waveform sequences of unknown modulation schemes without an additional timing synchronization module. The typical bidirectional LSTM (BiLSTM) structures are employed to tackle the continuous transmission issue. Moreover, complementary modulation classification layers are trained to sieve the valid bits of multiple modulation schemes. Thus, the whole process of our method can be deemed as multi-task learning. Simulation results reveal that the NN receiver can approach the bit error rate (BER) performance of theoretical BER values in the ideal channel. We further extend the method to harsh conditions with poor transceiver parameters and find that a certain gain can be obtained compared to traditional phase-lock-loop (PLL) based methods. The whole architecture is an effective joint synchronization and detection method, eliminating the complicated selection of appropriate algorithms for multiple modulation schemes, which further enhances the communication receiver’s general intelligence level.
20 July 2023 in IEEE Communications Letters
In this paper, we consider exploiting the natural redundancy (NR) that widely exists in the transmitted sources to improve the performance of communication signal detection. To exploit the abundant NR in the sources, we introduce the pre-trained models (PTMs) with a powerful capability to capture knowledge from massive labeled and unlabeled data into sequence detection. Especially for uncompressed Chinese text sources, we propose an effective sequence detection scheme aided by the widely used Bidirectional Encoder Representation from Transformers (BERT) deep learning model and combined with the beam search (BS) algorithm, which can achieve approximate sequence MAP estimation with an acceptable complexity. Simulation results demonstrate that the bit error rate (BER) performance of our proposed BERT-aided BS scheme for sequence detection significantly outperforms the classical detector.
19 December 2022 in IEEE Communications Letters
In this paper, we consider improving the performance of symbol detection by utilizing the natural redundancy (NR) that widely exists in the transmission sources. As an initial study, the NR considered in this paper is the redundancy inherent in the data caused by the non-uniform distribution of the memoryless transmission sources. We first theoretically analyze the utilization of NR to improve the performance of symbol detection from the perspective of information theory, and then propose a deep learning (DL) based iterative symbol detection algorithm with the source’s prior distribution unknown. Simulation results demonstrate that the performance of symbol detection can be improved by utilizing NR and our proposed DL-based iterative symbol detection algorithm can obtain the theoretically optimal performance even when the source distribution is blind to the detector.