We consider a dynamic multichannel access problem, where multiple correlated channels follow an unknown joint Markov model and users select the channel to transmit data. The objective is to find a policy that maximizes the expected long-term number of successful transmissions. The problem is formulated as a partially observable Markov decision process with unknown system dynamics. To overcome the challenges of unknown dynamics and prohibitive computation, we apply the concept of reinforcement learning and implement a deep Q-network (DQN). We first study the optimal policy for fixedpattern channel switching with known system dynamics and show through simulations that DQN can achieve the same optimal performance without knowing the system statistics. We then compare the performance of DQN with a Myopic policy and a Whittle Index-based heuristic through both more general simulations as well as real data trace and show that DQN achieves near-optimal performance in more complex situations. Finally, we propose an adaptive DQN approach with the capability to adapt its learning in time-varying scenarios.

We consider a multi-channel opportunistic communication system where the states of these channels evolve as independent and statistically identical Markov chains (the Gilbert- Elliot channel model). A user chooses one channel to sense and access in each slot and collects a reward determined by the state of the chosen channel. The problem is to design a sensing policy for channel selection to maximize the average reward, which can be formulated as a multi-arm restless bandit process. In this paper, we study the structure, optimality, and performance of the myopic sensing policy. We show that the myopic sensing policy has a simple robust structure that reduces channel selection to a round-robin procedure and obviates the need for knowing the channel transition probabilities. The optimality of this simple policy is established for the two-channel case and conjectured for the general case based on numerical results. The performance of the myopic sensing policy is analyzed, which, based on the optimality of myopic sensing, characterizes the maximum throughput of a multi-channel opportunistic communication system and its scaling behavior with respect to the number of channels. These results apply to cognitive radio networks, opportunistic transmission in fading environments, downlink scheduling in centralized networks, and resource-constrained jamming and anti-jamming.

A specific characteristic of sensor network applications is that the major traffic consists of data collection from various sensor source nodes to a sink via a unidirectional tree. In this paper, we propose DMAC, an energy efficient and low latency MAC that is designed and optimized for such data gathering trees in wireless sensor networks. We first show that previously proposed MAC protocols for sensor networks that utilize activation/sleep duty cycles suffer from a data forwarding interruption problem, whereby not all nodes on a multihop path to the sink can be notified of data delivery in progress, resulting in significant sleep delay. DMAC is designed to solve the interruption problem, by giving the active/sleep schedule of a node an offset that depends upon its depth on the tree. This scheme allows continuous packet forwarding because all nodes on the multihop path can be notified of the data delivery in progress. DMAC also adjusts node duty cycles adaptively according to the traffic load in the network by varying the number of active slots in an schedule interval. We further propose a data prediction mechanism and the use of more to send (MTS) packets in order to alleviate problems pertaining to channel contention and collisions. Our simulation results as well as experimental results with the Mote platform show that by exploiting the application‐specific structure of data gathering trees in sensor networks, DMAC provides significant energy savings and latency reduction while ensuring high data reliability. Copyright © 2007 John Wiley & Sons, Ltd.

In a typical Internet of Things (IoT) application where a central controller collects status updates from multiple terminals, e.g., sensors and monitors, through a wireless multiaccess uplink, an important problem is how to attain timely status updates autonomously. In this paper, the timeliness of the status is measured by the recently proposed age-of-information (AoI) metric; both the theoretical and practical aspects of the problem are investigated: we aim to obtain a scheduling policy with minimum AoI and, meanwhile, requires little signaling exchange overhead. Toward this end, we first consider the set of arrival-independent and renewal policies; the optimal policy thereof to minimize the time-average AoI is proved to be a round-robin policy with one-packet (latest packet only and others are dropped) buffers (RR-ONE). The optimality is established based on a generalized Poisson-arrival-see-time-average theorem. It is further proved that RR-ONE is asymptotically optimal among all policies in the massive IoT regime. The AoI steady-state stationary distribution under RR-ONE is also derived. An implementation scheme of RR-ONE is proposed which can accommodate dynamic terminal appearances with little overhead. In addition, considering scenarios where packets cannot be dropped, a Lyapunov optimization-based max-AoI-weight policy is proposed which achieves better performance compared with state-of-the-art.

As social media platforms evolve from text-based forums into multi-modal environments, the nature of misinformation in social media is also transforming accordingly. Taking advantage of the fact that visual modalities such as images and videos are more favorable and attractive to users, and textual content is sometimes skimmed carelessly, misinformation spreaders have recently targeted contextual connections between the modalities, e.g., text and image. Hence, many researchers have developed automatic techniques for detecting possible cross-modal discordance in web-based content. We analyze, categorize, and identify existing approaches in addition to the challenges and shortcomings they face to unearth new research opportunities in the field of multi-modal misinformation detection.

Base station sleeping is an effective way to reduce the energy consumption of mobile networks. Previous efforts to design sleeping control algorithms mainly rely on stochastic traffic models and analytical derivation. However the tractability of models often conflicts with the complexity of real-world traffic, making it difficult to apply in reality. In this paper, we propose a data-driven algorithm for dynamic sleeping control called DeepNap. This algorithm uses a Deep Q-network (DQN) to learn effective sleeping policies from high-dimensional raw observations or un-quantized systems state vectors. We propose to enhance the original DQN algorithm with action-wise experience replay and adaptive reward scaling to deal with the challenges in non-stationary traffic. We also provide a model-assisted variant of DeepNap through the Dyna framework for inferring and simulating system dynamics. Periodical traffic modeling makes it possible to capture the non-stationarity in real-world traffic and the incorporation with DQN allows for feature learning and generalization from model outputs. Experiments show that both the end-to-end and the model-assisted version of DeepNap outperform table-based Q-learning algorithm and the non-stationarity enhancements improve the stability of vanilla DQN.