Deep reinforcement learning (DRL) has emerged as a promising technique for optimizing the deployment of unmanned aerial vehicles (UAVs) for data collection in wireless sensor networks (WSNs) and Internet of Things (IoT) applications. With DRL, UAV trajectory can be optimized, optimal data collection points can be determined, sensor node transmissions can be scheduled efficiently, and irregular traffic patterns can be learned effectively. In view of the significance of DRL for UAV-assisted IoT research in general and, more specifically, its use for time-critical applications, this paper presents a review of the existing literature on UAV-aided data collection for WSN and IoT applications related to the application of DRL to minimize the Age of Information (AoI), a recent metric used to measure the degree of freshness of transmitted information collected in data-gathering applications. This review aims to provide insights into the state-of-the-art techniques, challenges, and opportunities in this domain through an extensive analysis of a sizable range of related research papers in this domain. It discusses application areas of UAV-assisted IoT, such as environmental monitoring, infrastructure inspection, and disaster response. Then, the paper focuses on the proposed works, their optimization objectives, architectures, simulation libraries and complexities of the various DRL-based approaches used. Thereafter discussion, challenges, and some opportunities for future work are provided. The findings of this review serve as a valuable resource for researchers and practitioners, guiding further advancements and innovations in the field of DRL for UAV-aided data collection in WSN and IoT applications.

Due to the limitations of sensor devices, including short transmission distance and constrained energy, unmanned aerial vehicles (UAVs) have been recently deployed to assist these nodes in transmitting their data. The sensor nodes (SNs) in wireless sensor networks (WSNs) or Internet of Things (IoT) networks periodically transmit their sensed data to UAVs to be relayed to the base station (BS). UAVs have been widely deployed in time-sensitive or real-time applications, such as in disaster areas, due to their ability to transmit data to the destination within a very short time. However, timely delivery of information by UAVs in WSN/IoT networks can be very complex due to various technical challenges, such as flight and trajectory control, as well as considerations of the scheduling of UAVs and SNs. Recently, the Age of Information (AoI), a metric used to measure the degree of freshness of information collected in data-gathering applications, has gained much attention. Numerous studies have proposed solutions to overcome the above-mentioned challenges, including adopting several optimization and machine learning (ML) algorithms for diverse architectural setups to minimize the AoI. In this paper, we conduct a systematic literature review (SLR) to study past literature on age minimization in UAV-assisted data-gathering architecture to determine the most important design components. Three crucial design aspects in AoI minimization were discovered from analyzing the 26 selected articles, which focused on energy management, flight trajectory, and UAV/SN scheduling. We also investigate important issues related to these identified design aspects, for example, factors influencing energy management, including the number of visited sensors, energy levels, UAV cooperation, flight time, velocity control, and charging optimization. Issues related to flight trajectory and sensor node scheduling are also discussed. In addition, future considerations on problems such as traffic prioritization, packet delivery errors, system optimization, UAV-to-sensor node association, and physical impairments are also identified.