Motivated by the need to ensure timely delivery of information (e.g., status updates) in the Internet-of-things (IoT) paradigm, this paper investigates the role of an Unmanned aerial vehicle (UAV) as a mobile relay to minimize the average Peak Age-of-information (PAoI) for a source-destination pair. For this setup, we formulate an optimization problem to jointly optimize the UAV's flight trajectory as well as energy and service time allocations for packet transmissions. In order to solve this non-convex problem, we propose an efficient iterative algorithm and establish its convergence analytically. Closed-form solutions for some sub-problems are also provided. One of the sub-problems we solve in this procedure is to jointly optimize the energy and service time allocations for a given trajectory of the UAV. This problem is of interest on its own right because in some cases we may not be able to alter the UAV's trajectory based on the locations of the IoT devices (especially when its primary mission is something else). Our numerical results quantify the gains that can be achieved by additionally optimizing the UAV's trajectory.

In this paper, we study a real-time monitoring system in which multiple source nodes are responsible for sending update packets to a common destination node in order to maintain the freshness of information at the destination. Since it may not always be feasible to replace or recharge batteries in all source nodes, we consider that the nodes are powered through wireless energy transfer (WET) by the destination. For this system setup, we investigate the optimal online sampling policy (referred to as the age-optimal policy) that jointly optimizes WET and scheduling of update packet transmissions with the objective of minimizing the long-term average weighted sum of Age of Information (AoI) values for different physical processes (observed by the source nodes) at the destination node, referred to as the sum-AoI. To solve this optimization problem, we first model this setup as an average cost Markov decision process (MDP) with finite state and action spaces. Due to the extreme curse of dimensionality in the state space of the formulated MDP, classical reinforcement learning algorithms are no longer applicable to our problem even for reasonable-scale settings. Motivated by this, we propose a deep reinforcement learning (DRL) algorithm that can learn the age-optimal policy in a computationally-efficient manner. We further characterize the structural properties of the age-optimal policy analytically, and demonstrate that it has a threshold-based structure with respect to the AoI values for different processes. We extend our analysis to characterize the structural properties of the policy that maximizes average throughput for our system setup, referred to as the throughputoptimal policy. Afterwards, we analytically demonstrate that the structures of the age-optimal and throughput-optimal policies are different. We also numerically demonstrate these structures as well as the impact of system design parameters on the optimal achievable average weighted sum-AoI.

In this article, an unmanned aerial vehicle (UAV)-assisted wireless network is considered in which a battery-constrained UAV is assumed to move towards energy-constrained ground nodes to receive status updates about their observed processes. The UAV's flight trajectory and scheduling of status updates are jointly optimized with the objective of minimizing the normalized weighted sum of Age of Information (NWAoI) values for different physical processes at the UAV. The problem is first formulated as a mixed-integer program. Then, for a given scheduling policy, a convex optimization-based solution is proposed to derive the UAV's optimal flight trajectory and time instants on updates. However, finding the optimal scheduling policy is challenging due to the combinatorial nature of the formulated problem. Therefore, to complement the proposed convex optimization-based solution, a finite-horizon Markov decision process (MDP) is used to find the optimal scheduling policy. Since the state space of the MDP is extremely large, a novel neural combinatorial-based deep reinforcement learning (NCRL) algorithm using deep Q-network (DQN) is proposed to obtain the optimal policy. However, for large-scale scenarios with numerous nodes, the DQN architecture cannot efficiently learn the optimal scheduling policy anymore. Motivated by this, a long short-term memory (LSTM)-based autoencoder is proposed to map the state space to a fixed-size vector representation in such large-scale scenarios while capturing the spatio-temporal interdependence between the update locations and time instants. A lower bound on the minimum NWAoI is analytically derived which provides system design guidelines on the appropriate choice of importance weights for different nodes. Furthermore, an upper bound on the UAV's minimum speed is obtained to achieve this lower bound value. The numerical results also demonstrate that the proposed NCRL approach can significantly improve the achievable NWAoI per process compared to the baseline policies, such as weight-based and discretized state DQN policies.

This paper studies the interplay between device-to-device (D2D) communications and real-time monitoring systems in a cellular-based Internet of Things (IoT) network. In particular, besides the possibility that the IoT devices communicate directly with each other in a D2D fashion, we consider that they frequently send time-sensitive information/status updates (about some underlying physical processes observed by them) to their nearest cellular base stations (BSs). Specifically, we model the locations of the IoT devices as a bipolar Poisson Point Process (PPP) and that of the BSs as another independent PPP. For this setup, we characterize the performance of D2D communications using the average network throughput metric whereas the performance of the real-time applications is quantified by the Age of Information (AoI) metric. The IoT devices are considered to employ a distance-proportional fractional power control scheme while sending status updates to their serving BSs. Hence, depending upon the maximum transmission power available, the IoT devices located within a certain distance from the BSs can only send status updates. This association strategy, in turn, forms the Johnson-Mehl (JM) tessellation, such that the IoT devices located in the JM cells are allowed to send status updates. The average network throughput is obtained by deriving the mean success probability for the D2D links. On the other hand, the temporal mean AoI of a given status update link can be treated as a random variable over space since its success delivery rate is a function of the interference field seen from its receiver. Thus, in order to capture the spatial disparity in the AoI performance, we characterize the spatial moments of the temporal mean AoI. In particular, we obtain these spatial moments by deriving the moments of both the conditional success probability and the conditional scheduling probability for status update links. Our results provide useful design guidelines on the efficient deployment of future massive IoT networks that will jointly support D2D communications and several cellular network-enabled real-time applications.

This paper considers a large-scale wireless network consisting of source-destination (SD) pairs, where the sources send time-sensitive information, termed status updates, to their corresponding destinations in a time-slotted fashion. We employ age of information (AoI) for quantifying the freshness of the status updates measured at the destination nodes under the preemptive and non-preemptive queueing disciplines with no storage facility. The non-preemptive queue drops the newly arriving updates until the update in service is successfully delivered, whereas the preemptive queue replaces the current update in service with the newly arriving update, if any. As the update delivery rate for a given link is a function of the interference field seen from the receiver, the temporal mean AoI can be treated as a random variable over space. Our goal in this paper is to characterize the spatial distribution of the mean AoI observed by the SD pairs by modeling them as a bipolar Poisson point process (PPP). Towards this objective, we first derive accurate bounds on the moments of success probability while efficiently capturing the interference-induced coupling in the activities of the SD pairs. Using this result, we then derive tight bounds on the moments as well as the spatial distribution of peak AoI (PAoI). Our numerical results verify our analytical findings and demonstrate the impact of various system design parameters on the mean PAoI.