Communication systems to date primarily aim at reliably communicating bit sequences. Such an approach provides efficient engineering designs that are agnostic to the meanings of the messages or to the goal that the message exchange aims to achieve. Next generation systems, however, can be potentially enriched by folding message semantics and goals of communication into their design. Further, these systems can be made cognizant of the context in which communication exchange takes place, thereby providing avenues for novel design insights. This tutorial summarizes the efforts to date, starting from its early adaptations, semantic-aware and task-oriented communications, covering the foundations, algorithms and potential implementations. The focus is on approaches that utilize information theory to provide the foundations, as well as the significant role of learning in semantics and task-aware communications.

This tutorial paper focuses on the variants of the bottleneck problem taking an information theoretic perspective and discusses practical methods to solve it, as well as its connection to coding and learning aspects. The intimate connections of this setting to remote source-coding under logarithmic loss distortion measure, information combining, common reconstruction, the Wyner–Ahlswede–Korner problem, the efficiency of investment information, as well as, generalization, variational inference, representation learning, autoencoders, and others are highlighted. We discuss its extension to the distributed information bottleneck problem with emphasis on the Gaussian model and highlight the basic connections to the uplink Cloud Radio Access Networks (CRAN) with oblivious processing. For this model, the optimal trade-offs between relevance (i.e., information) and complexity (i.e., rates) in the discrete and vector Gaussian frameworks is determined. In the concluding outlook, some interesting problems are mentioned such as the characterization of the optimal inputs (“features”) distributions under power limitations maximizing the “relevance” for the Gaussian information bottleneck, under “complexity” constraints.

The problem of distributed representation learning is one in which multiple observations $X_1,\ldots,X_K$ are processed separately to learn as much information as possible about some source $Y$ . We investigate this problem from information-theoretic grounds, through a generalization of Tishby's centralized Information Bottleneck (IB) method to the distributed setting. Specifically, $K\geq 2$ encoders, compress their observations $X_1,\ldots,X_K$ separately such that, collectively, the produced representations preserve as much information as possible about $Y$ . We study both discrete memoryless (DM) and vector Gaussian data models. For the discrete model, we establish a single-letter characterization of the optimal tradeoff for a class of memoryless sources. For the vector Gaussian model, we provide an explicit characterization of the optimal complexity-relevance tradeoff. Furthermore, we develop a variational bound on the complexity-relevance tradeoff which generalizes the evidence lower bound (ELBO) to the distributed setting. We provide two algorithms to compute this bound: i) a Blahut-Arimoto type iterative algorithm which computes optimal complexity-relevance mappings by iterating over a set of self-consistent equations, and ii) a variational inference type algorithm in which the encoding mappings are parametrized by neural networks, the bound approximated by Markov sampling and optimized with stochastic gradient descent. Numerical results on synthetic and real datasets are provided to support the efficiency of the approaches and algorithms developed in this paper.

We study the transmission over a network in which users send information to a remote destination through relay nodes that are connected to the destination via finite-capacity error-free links, i.e., a cloud radio access network. The relays are constrained to operate without knowledge of the users’ codebooks, i.e., they perform oblivious processing. The destination, or central processor, however, is informed about the users’ codebooks. We establish a single-letter characterization of the capacity region of this model for a class of discrete memoryless channels in which the outputs at the relay nodes are independent given the users’ inputs. We show that both relaying à-la Cover-El Gamal, i.e., compress-and-forward with joint decompression and decoding, and “noisy network coding”, are optimal. The proof of the converse part establishes, and utilizes, connections with the Chief Executive Officer (CEO) source coding problem under logarithmic loss distortion measure. Extensions to general discrete memoryless channels are also investigated. In this case, we establish inner and outer bounds on the capacity region. For memoryless Gaussian channels within the studied class of channels, we characterize the capacity region when the users are constrained to time-share among Gaussian codebooks. Furthermore, we also discuss the suboptimality of separate decompression-decoding and the role of time-sharing.

Background: Against a long-term trend of increasing demand, the COVID-19 pandemic has led to a global rise in common mental disorders. Now more than ever, there is an urgent need for scalable, evidence-based interventions to support mental well-being. Objective: The aim of this proof-of-principle study was to evaluate the efficacy of a mobile-based app in adults with self-reported symptoms of anxiety and stress in a randomized control trial that took place during the first wave of the COVID-19 pandemic in the United Kingdom. Methods: Adults with mild to severe anxiety and moderate to high levels of perceived stress were randomized to either the intervention or control arm. Participants in the intervention arm were given access to the Foundations app for the duration of the 4-week study. All participants were required to self-report a range of validated measures of mental well-being (10-item Connor-Davidson Resilience scale [CD-RISC-10], 7-item Generalized Anxiety Disorder scale [GAD-7], Office of National Statistics Four Subjective Well-being Questions [ONS-4], World Health Organization-5 Well-Being Index [WHO-5]) and sleep (Minimal Insomnia Scale [MISS]) at baseline and at weeks 2 and 4. The self-reported measures of perceived stress (10-item Perceived Stress Score [PSS-10]) were obtained weekly. Results: A total of 136 participants completed the study and were included in the final analysis. The intervention group (n=62) showed significant improvements compared to the control group (n=74) on measures of anxiety, with a mean GAD-7 score change from baseline of –1.35 (SD 4.43) and –0.23 (SD 3.24), respectively (t134=1.71, P=.04); resilience, with a mean change in CD-RISC score of 1.79 (SD 4.08) and –0.31 (SD 3.16), respectively (t134=–3.37, P<.001); sleep, with a mean MISS score change of –1.16 (SD 2.67) and –0.26 (SD 2.29), respectively (t134=2.13, P=.01); and mental well-being, with a mean WHO-5 score change of 1.53 (SD 5.30) and –0.23 (SD 4.20), respectively (t134=–2.16, P=.02), within 2 weeks of using Foundations, with further improvements emerging at week 4. Perceived stress was also reduced within the intervention group, although the difference did not reach statistical significance relative to the control group, with a PSS score change from baseline to week 2 of –2.94 (SD 6.84) and –2.05 (SD 5.34), respectively (t134= 0.84, P=.20). Conclusions: This study provides a proof of principle that the digital mental health app Foundations can improve measures of mental well-being, anxiety, resilience, and sleep within 2 weeks of use, with greater effects after 4 weeks. Foundations therefore offers potential as a scalable, cost-effective, and accessible solution to enhance mental well-being, even during times of crisis such as the COVID-19 pandemic. Trial Registration: OSF Registries osf.io/f6djb; https://osf.io/vm3xq

We study the transmission over a cloud radio access network in which multiple base stations, acting as relay nodes, are connected to a central processor (CP) via error-free ratelimited backhaul links. We propose two lattice-based coding schemes. In the first scheme, each relay node decodes linear combinations of the users’ messages in the spirit of computeand- forward (CoF), but departs from it essentially in that the decoded equations are remapped to equations on the users’ input symbols, sent compressed in a lossy manner to the CP, and are not required to be linearly independent. The compression accounts for the correlation between equations at the relay nodes through Wyner-Ziv coding. Also, by opposition to the standard CoF, an appropriate multi-user decoder is utilized to recover the sent messages. The second scheme generalizes the first one by also allowing, at each relay node, a joint compression of the decoded equation on the users’ input symbols and the received signal. Both schemes apply in general, but are especially suited for situations in which there are more users that relays. We show that both schemes can outperform standard CoF and successive Wyner- Ziv (SWZ) schemes in certain regimes, and illustrate the gains through some numerical examples.