The Information Bottleneck method is a powerful and generic tool from the field of machine learning. It compresses an observation to a quantized variable while attempting to preserve the mutual information shared with a relevant random variable. This paper describes a new application of the Information Bottleneck method in communications. It explains in detail, how the Information Bottleneck method can be applied to construct discrete message passing decoders for regular lowdensity parity-check codes. The obtained decoders process only unsigned integers and use only simple lookup tables as node operations. As a consequence, the decoders can be implemented using only unsigned integer arithmetic which makes them significantly simpler and faster than state-of-the-art decoders which process real valued log-likelihood ratios. Anyway, included results show that the considered discrete message passing decoders perform surprisingly close to optimum message passing decoders and even outperform state-of-the-art decoders such as the minsum decoder. We aim to take the reader on a journey from the theoretical idea of the Information Bottleneck method to a complete design framework for the considered discrete decoders. Several included figures and examples illustrate the decoder construction process and its analysis.

In high-throughput applications, low-complexity and low-latency channel decoders are inevitable. Hence, for low-density parity-check (LDPC) codes, message passing decoding has to be implemented with coarse quantization—that is, the exchanged beliefs are quantized with a small number of bits. This can result in a significant performance degradation with respect to decoding with high-precision messages. Recently, so-called information-bottleneck decoders were proposed which leverage a machine learning framework (i.e., the information bottleneck method) to design coarse-precision decoders with error-correction performance close to high-precision belief-propagation decoding. In these decoders, all conventional arithmetic operations are replaced by look-up operations. Irregular LDPC codes for next-generation fiber optical communication systems are characterized by high code rates and large maximum node degrees. Consequently, the implementation complexity is mainly influenced by the memory required to store the look-up tables. In this paper, we show that the complexity of information-bottleneck decoders remains manageable for irregular LDPC codes if our proposed construction approach is deployed. Furthermore, we reveal that in order to design information bottleneck decoders for arbitrary degree distributions, an intermediate construction step which we call message alignment has to be included. Exemplary numerical simulations show that incorporating message alignment in the construction yields a 4-bit information bottleneck decoder which performs only $0.15$ dB worse than a double-precision belief propagation decoder and outperforms a min-sum decoder.

Receiver implementations with very low quantization resolution will play an important role in 5G, as high precision quantization and signal processing are costly in terms of computational resources and chip area. Therefore, low resolution receivers with quasi optimum performance will be required to meet complexity and latency constraints. The Information Bottleneck method allows for a novel, information centric approach to design such receivers. The method was originally introduced by Naftali Tishby et al. and mostly used in the machine learning field so far. Interestingly, it can also be applied to build surprisingly good digital communication receivers which work fundamentally different than state-of-the-art receivers. Instead of minimizing the quantization error, receiver components with maximum preservation of relevant information for a given bit width can be designed. All signal processing in the resulting receivers is performed using only simple lookup operations. In this paper, we first provide a brief introduction to the design of receiver components with the Information Bottleneck method. We keep referring to decoding of low-density parity-check codes as a practical example. The focus of the paper lies on practical decoder implementations on a digital signal processor which illustrate the potential of the proposed technique. An Information Bottleneck decoder with 4 bit message passing decoding is found to outperform 8 bit implementations of the well-known min-sum decoder in terms of bit error rate and to perform extremely close to an 8 bit belief propagation decoder, while offering considerably higher net decoding throughput than both conventional decoders.

The Information Bottleneck method allows to construct information-optimum message passing decoders for low-density parity-check codes. In such decoders lookup tables replace the classical node operations of the variable and the check nodes. These lookup tables are designed using the Information Bottleneck principle of maximizing the relevant information. Unlike state-of-the-art decoders which use real valued log-likelihood ratios for decoding, the considered decoders do not process any real values, but only quantization indices. Nevertheless, they have performance extremely close to belief propagation decoding. Since hardware representation of unsigned integers is efficient and lookup table implementations have low complexity, it is reasonable to assume that the designed decoders offer advantages over their conventional counterparts in practice. In this paper, we evaluate, quantify and discuss these advantages in a practical experiment. Our focus lies on a software defined radio application, where the channel decoder is implemented on a digital signal processor. We present several implementations of the considered decoders and compare them with state-of-the-art decoders. Our results show considerable gains of the Information Bottleneck decoders in terms of bit error rate performance and net decoding throughput.

Digital communication receivers extract information about the transmitted data from the received signal in subsequent processing steps, such as synchronization, demodulation and channel decoding. Technically, the receiver-side signal processing for conducting these tasks is complex and hence causes bottleneck situations in terms of power, delay and chip area. Typically, many bits per sample are required to represent and process the received signal in the digital receiver hardware accurately. In addition, demanding arithmetical operations are required in the signal processing algorithms. A popular recent trend is designing entire receiver chains or some of their crucial building blocks from an information theoretical perspective. Signal processing blocks with very simple mathematical operations can be designed to directly maximize the relevant information that flows through them. At the same time, a strong quantization reduces the number of bits processed in the receiver to further lower the complexity. The described system design approach follows the principle of the information bottleneck method. Different authors proposed various ideas to design and implement mutual information-maximizing signal processing units. The first important aim of this article is to explain the fundamental similarities between the information bottleneck method and the functionalities of communication receivers. Based on that, we present and investigate new results on an entire receiver chain that is designed following the information bottleneck design principle. Afterwards, we give an overview of different techniques following the information bottleneck design paradigm from the literature, mainly dealing with channel decoding applications. We analyze the similarities of the different approaches for information bottleneck signal processing. This comparison leads to a general view on information bottleneck signal processing which goes back to the learning of parameters of trainable functions that maximize the relevant mutual information under compression.