Freshness of status update packets is essential for enabling services where a destination needs the most recent measurements of various sensors. In this paper, we study the information freshness of single-server multi-source queueing models under a first-come first-served (FCFS) serving policy. In the considered model, each source independently generates status update packets according to a Poisson process. The information freshness of the status updates of each source is evaluated by the average age of information (AoI). We derive an exact expression for the average AoI for the case with exponentially distributed service time, i.e., for a multi-source M/M/1 queueing model. Moreover, we derive three approximate expressions for the average AoI for a multi-source M/G/1 queueing model having a general service time distribution. Simulation results are provided to validate the derived exact average AoI expression, to assess the tightness of the proposed approximations, and to demonstrate the AoI behavior for different system parameters.

Information freshness is crucial for time-critical IoT applications, e.g., monitoring and control. We consider an IoT status update system with users, energy harvesting sensors, and a cache-enabled edge node. The users receive time-sensitive information about physical quantities, each measured by a sensor. Users demand for the information from the edge node whose cache stores the most recently received measurements from each sensor. To serve a request, the edge node either commands the sensor to send an update or retrieves the aged measurement from the cache. We aim at finding the best actions of the edge node to minimize the average AoI of the served measurements at the users, termed on-demand AoI. We model this problem as a Markov decision process and develop reinforcement learning (RL) algorithms: model-based value iteration and model-free Q-learning. We also propose a Q-learning method for the realistic case where the edge node is informed about the sensors’ battery levels only via the status updates. The case under transmission limitations is also addressed. Furthermore, properties of an optimal policy are characterized. Simulation results show that an optimal policy is a threshold-based policy and that the proposed RL methods significantly reduce the average cost compared to several baselines.

We study the information freshness under three different source aware packet management policies in a status update system consisting of two independent sources and one server. The packets of each source are generated according to the Poisson process and the packets are served according to an exponentially distributed service time. We derive the average age of information (AoI) of each source using the stochastic hybrid systems (SHS) technique for each packet management policy. In Policy 1, the queue can contain at most two waiting packets at the same time (in addition to the packet under service), one packet of source 1 and one packet of source 2. When the server is busy at an arrival of a packet, the possible packet of the same source waiting in the queue (hence, source-aware) is replaced by the arrived fresh packet. In Policy 2, the system (i.e., the waiting queue and the server) can contain at most two packets, one from each source. When the server is busy at an arrival of a packet, the possible packet of the same source in the system is replaced by the fresh packet. Policy 3 is similar to Policy 2 but it does not permit preemption in service, i.e., while a packet is under service all new arrivals from the same source are blocked and cleared. Numerical results are provided to assess the fairness between sources and the sum average AoI of the proposed policies.

We consider a resource-constrained IoT network, where multiple users make on-demand requests to a cache-enabled edge node to send status updates about various random processes, each monitored by an energy harvesting sensor. The edge node serves users’ requests by deciding whether to command the corresponding sensor to send a fresh status update or retrieve the most recently received measurement from the cache. Our objective is to find the best actions of the edge node to minimize the average age of information (AoI) of the received measurements upon request, i.e., average on-demand AoI , subject to per-slot transmission and energy constraints . First, we derive a Markov decision process model and propose an iterative algorithm that obtains an optimal policy. Then, we develop an asymptotically optimal low-complexity algorithm – termed relax-then-truncate – and prove that it is optimal as the number of sensors goes to infinity. Simulation results illustrate that the proposed relax-then-truncate approach significantly reduces the average on-demand AoI compared to a request-aware greedy policy and a weighted AoI policy, and also depict that it performs close to the optimal solution even for moderate numbers of sensors.

We consider a status update system where multiple sensors communicate timely information about various random processes to a sink. The sensors share orthogonal sub-channels to transmit such information in the form of status update packets. A central controller can control the sampling actions of the sensors to trade-off between the transmit power consumption and information freshness which is quantified by the Age of Information (AoI). We jointly optimize the sampling action of each sensor, the transmit power allocation, and the sub-channel assignment to minimize the average total transmit power of all sensors, subject to a maximum average AoI constraint for each sensor. To solve the problem, we develop a dynamic control algorithm using the Lyapunov drift-plus-penalty method and provide optimality analysis of the algorithm. According to the Lyapunov drift-plus-penalty method, to solve the main problem, we need to solve an optimization problem in each time slot which is a mixed integer non-convex optimization problem. We propose a low-complexity sub-optimal solution for this per-slot optimization problem that provides near-optimal performance and we evaluate the computational complexity of the solution. Numerical results illustrate the performance of the proposed dynamic control algorithm and the performance of the sub-optimal solution for the per-slot optimization problem versus the different parameters of the system. The results show that the proposed dynamic control algorithm achieves more than 60% saving in the average total transmit power compared to a baseline policy.

We consider a multi-source relaying system where independent sources randomly generate status update packets which are sent to the destination with the aid of a relay through unreliable links. We develop transmission scheduling policies to minimize the weighted sum average age of information (AoI) subject to transmission capacity and long-run average resource constraints. We formulate a stochastic control optimization problem and solve it using a constrained Markov decision process (CMDP) approach and a drift-plus-penalty method. The CMDP problem is solved by transforming it into an MDP problem using the Lagrangian relaxation method. We theoretically analyze the structure of optimal policies for the MDP problem and subsequently propose a structure-aware algorithm that returns a practical near-optimal policy. Using the drift-plus-penalty method, we devise a near-optimal low-complexity policy that performs the scheduling decisions dynamically. We also develop a model-free deep reinforcement learning policy for which the Lyapunov optimization theory and a dueling double deep Q-network are employed. The complexities of the proposed policies are analyzed. Simulation results are provided to assess the performance of our policies and validate the theoretical results. The results show up to 91% performance improvement compared to a baseline policy.