A large body of applications that involve monitoring, decision making, and forecasting require timely status updates for their efficient operation. Age of Information (AoI) is a newly proposed metric that effectively captures this requirement. Recent research on the subject has derived AoI optimal policies for the generation of status updates and AoI optimal packet queueing disciplines. Unlike previous research we focus on low-end devices that typically support monitoring applications in the context of the Internet of Things. We acknowledge that these devices host a diverse set of applications some of which are AoI sensitive while others are not. Furthermore, due to their limited computational resources they typically utilize a simple First-In First-Out (FIFO) queueing discipline. We consider the problem of optimally controlling the status update generation process for a system with a source-destination pair that communicates via a wireless link, whereby the source node is comprised of a FIFO queue and serves two applications, one that is AoI sensitive and one that is not. We formulate this problem as a dynamic programming problem and utilize the framework of Markov Decision Processes to derive the optimal policy for the generation of status update packets. Due to the lack of comparable methods in the literature we compare the derived optimal policies against baseline policies, such as the zero-wait policy. Results indicate that baseline policies fail to capture the complex system dynamics which determine the relationship between the frequency of status update generation and the resulting queueing delay and thus perform poorly. To the best of our knowledge, the derived optimal policy does not exhibit a simple structure; thus, we utilized the baseline policies, whose operation is intuitive, to gain insight into the inner workings of the optimal policy.

With its growing number of deployed devices and applications, the Internet of Things (IoT) raises significant challenges for network maintenance procedures. In this work, we address a problem of active fault detection in an IoT scenario, whereby a monitor can probe a remote device to acquire fresh information and facilitate fault detection. However, probing could significantly impact the system’s energy and communication resources. To this end, we utilize the Age of Information to measure the freshness of information at the monitor and adopt a semantics-aware communication approach between the monitor and the remote device. In semantics-aware communications, the processes of generating and transmitting information are treated jointly to consider the importance of information and the purpose of communication. We formulate the problem as a Partially Observable Markov Decision Process and show analytically that the optimal policy is of a threshold type. Finally, we use a computationally efficient stochastic approximation algorithm to approximate the optimal policy and present numerical results that exhibit the advantage of our approach compared to a conventional delay-based probing policy.

We address the problem of estimating the end-to-end throughput of UDP flows in a multi-radio, multi-channel, and multi-rate 802.11 wireless mesh network (WMN). Estimates of the end-to-end throughput of flows in a WMN can facilitate the design of traffic-aware routing and channel assignment algorithms. We introduce a simple and intuitive model that captures key features of 802.11 WMNs such as contention, interference, and performance degradation due to links with low transmission rates. We use simulation to validate the accuracy of the proposed model in various scenarios for a network topology containing multiple transmitters and wired gateways.

In this study, we investigate the optimal transmission policies within an energy harvesting status update system, where the demand for status updates depends on the state of the source. The system monitors a source with two contextual states characterized by a Markovian stochastic process, which can be in either a normal state or an alarm state, with a higher demand for fresh updates when the source is in the alarm state. We propose a metric to capture the freshness of status updates for each state of the stochastic process by introducing two Age of Information (AoI) variables, extending the definition of AoI to account for the state changes of the stochastic process. We formulate the problem as a Markov Decision Process (MDP), utilizing a transition cost function that applies linear and non-linear penalties based on AoI and the state of the stochastic process. Through analytical investigation, we delve into the structure of the optimal transmission policy for the resulting MDP problem. Furthermore, we evaluate the derived policies via numerical results and demonstrate their effectiveness in reserving energy in anticipation of forthcoming alarm states.

The Internet of things (IoT) sets new bandwidth requirements for devices' seamless connectivity and calls for new communication paradigms such as opportunistic spectrum access. However, realization of opportunistic spectrum access without causing interference to primary users poses significant challenges that are further exacerbated by the constrained nature of IoT devices both in terms of computational resources and battery capacity. In this work we consider the problem of energy efficient collection of primary users' spectrum occupancy data that may be used to facilitate the prediction of future primary users' states. We propose a process for a cognitive IoT device to decide the optimal spectrum sensing period for each state of a primary user. The decision process considers both energy consumption and accuracy in the tracking of a primary user's state within the period. We model the problem at hand as a semi-Markov decision process embedded within a base Markov decision process using the framework of options. Finally, we utilize two variants of the Q-learning algorithm that are known to converge to optimal policies and we propose enhancements that significantly speed up the learning process. Numerical results verify the effectiveness of the proposed enhanced learning strategies.

The Internet-of-things facilitates the development of many groundbreaking applications. A large number of these applications involve mobile end nodes and a sparsely deployed network of base stations that operate as gateways to the Internet. Most of the mobile nodes, at least within city areas, are connected through low power wide area networking technologies (LPWAN) using public frequencies. Mobility and sparse network coverage result in long delays and intermittent connectivity for the end nodes. Disruption Tolerant Networks and utilization of heterogeneous wireless interfaces have emerged as key technologies to tackle the problem at hand. The first technology renders communication resilient to intermittent connectivity by storing and carrying data while the later increases the communication opportunities of the end nodes and at the same time reduces energy consumption whenever short-range communication is possible. However, one has to consider that end nodes are typically both memory and energy constrained devices which makes finding an energy efficient data transmission policy for heterogeneous disruption tolerant networks imperative. In this work we utilize information related to the spatial availability of network resources and localization information to formulate the problem at hand as a dynamic programming problem. Next, we utilize the framework of Markov Decision Processes to derive approximately optimal and suboptimal data transmission policies. We also prove that we can achieve improved packet transmission policies and reduce energy consumption, extending battery lifetime. This is achieved by knowing the spatial availability of heterogeneous network resources combined with the mobile node’s location information. Numerical resultsshow significant gains achieved by utilizing the derived approximately optimal and suboptimal policies.