Unmanned aerial vehicles (UAVs) are envisioned to play a key role in intelligent transportation systems to complement the communication infrastructure in future smart cities. UAV-assisted vehicular networking research typically adopts throughput and latency as the main performance metrics. These conventional metrics, however, are not adequate to reflect the freshness of the information, an attribute that has been recently identified as a critical requirement to enable services such as autonomous driving and accident prevention. In this paper, we consider a UAV-assisted single-hop vehicular network, wherein sensors (e.g., LiDARs and cameras) on vehicles generate time sensitive data streams, and UAVs are used to collect and process this data while maintaining a minimum age of information (AoI). We aim to jointly optimize the trajectories of UAVs and find scheduling policies to keep the information fresh under minimum throughput constraints. The formulated optimization problem is shown to be mixed integer non-linear program (MINLP) and generally hard to be solved. Motivated by the success of machine learning (ML) techniques particularly deep learning in solving complex problems with low complexity, we reformulate the trajectories and scheduling policies problem as a Markov decision process (MDP) where the system state space considers the vehicular network dynamics. Then, we develop deep reinforcement learning (DRL) to learn the vehicular environment and its dynamics in order to handle UAVs' trajectory and scheduling policy. In particular, we leverage Deep Deterministic Policy Gradient (DDPG) for learning the trajectories of the deployed UAVs to efficiently minimize the Expected Weighted Sum AoI (EWSA). Simulations results demonstrate the effectiveness of the proposed design and show the deployed UAVs adapt their velocities during the data collection mission in order to minimize the AoI.

Fifth generation wireless networks are expected to provide advanced capabilities and create new markets. Among the emerging markets, Internet of Things (IoT) use cases are standing out with the proliferation of a wide range of sensors that can be configured to continuously monitor and transmit data for intelligent processing and decision making. Devices in such scenarios are normally extremely energy-constrained and often exist in large numbers and can be located in hard-to-reach areas; the fact that necessitates the design and implementation of effective energy-aware data collection mechanisms. To this end, we propose the utilization of Unmanned Aerial Vehicles (UAVs) to collect data in dense wireless sensor networks (WSNs) using projection-based Compressive Data Gathering (CDG) as a novel solution methodology. CDG is utilized to aggregate data en-route from a large set of sensor nodes to selected projection nodes acting as cluster heads in order to reduce the number of needed transmissions leading to notable energy savings and extended network lifetime. The UAV transfers the gathered data from the cluster heads to a remote sink node, e.g., a 5G cellular base station, which avoids the need for long range transmissions or multihop communications among the sensors. Our problem definition aims at clustering the sensors, constructing an optimized forwarding tree per cluster, and gathering the data from selected cluster head nodes based on projection-based CDG with minimized UAV trajectory distance. We formulate a joint optimization problem and divide it into four complementary subproblems to generate close-to-optimal results with lower complexity. Moreover, we propose a set of effective algorithms to generate solutions for relatively large-scale network scenarios. We demonstrate the superiority of the proposed approach and the designed algorithms via detailed performance results with analysis, comparisons and insights.

Unmanned Aerial Vehicles (UAVs) are gaining growing interests due to the paramount roles they play, particularly these days, in enabling new services that help modernize our transportation, supply chain, search and rescue, among others. They are capable of positively influencing wireless systems through enabling and fostering emerging technologies such as autonomous driving, vertical industries, virtual reality and so many others. The Internet of Vehicles is a prime sector benefiting from the services offered by future cellular systems in general and UAVs in particular, and this paper considers the problem of content delivery to vehicles on road segments with either overloaded or no available communication infrastructure. Incoming vehicles demand service from a library of contents that is partially cached at the UAV; the content of the library is also assumed to change as new vehicles carrying more popular contents arrive. Each inbound vehicle makes a request and the UAV decides on its best trajectory to provide service while maximizing a certain operational utility. Given the energy limitation at the UAV, we seek an energy efficient solution. Hence, our problem consists of jointly finding caching decisions, UAV trajectory and radio resource allocation which is formulated mathematically as a Mixed Integer Non-Linear Problem (MINLP). However, owing to uncertainties in the environment (e.g., random arrival of vehicles, their requests for contents and their existing contents), it is often hard and impractical to solve using standard optimization techniques. To this end, we formulate our problem as a Markov Decision Process (MDP) and we resort to tools such as Proximal Policy Optimization (PPO), a very promising Reinforcement Learning method, along with a set of crafted algorithms to solve our problem. Finally, we conduct simulation-based experiments to analyze and demonstrate the superiority of our solution approach compared with four counterparts and baseline schemes.

Disaster management, search and rescue missions, and health monitoring are examples of critical applications that require object localization with high precision and sometimes in a timely manner. In the absence of the global positioning system (GPS), the radio received signal strength index (RSSI) can be used for localization purposes due to its simplicity and cost-effectiveness. However, due to the low accuracy of RSSI, unmanned aerial vehicles (UAVs) or drones may be used as an efficient solution for improved localization accuracy due to their agility and higher probability of line-of-sight (LoS). Hence, in this context, we propose a novel framework based on reinforcement learning (RL) to enable a UAV (agent) to autonomously find its trajectory that results in improving the localization accuracy of multiple objects in shortest time and path length, fewer signal-strength measurements (waypoints), and/or lower UAV energy consumption. In particular, we first control the agent through initial scan trajectory on the whole region to 1) know the number of nodes and estimate their initial locations, and 2) train the agent online during operation. Then, the agent forms its trajectory by using RL to choose the next waypoints in order to minimize the average location errors of all objects. Our framework includes detailed UAV to ground channel characteristics with an empirical path loss and log-normal shadowing model, and also with an elaborate energy consumption model. We investigate and compare the localization precision of our approach with existing methods from the literature by varying the UAV's trajectory length, energy, number of waypoints, and time. Furthermore, we study the impact of the UAV's velocity, altitude, hovering time, communication range, number of maximum RSSI measurements, and number of objects. The results show the superiority of our method over the state-of-art and demonstrates its fast reduction of the localization error.

The success in transitioning towards smart cities relies on the availability of information and communication technologies that meet the demands of this transformation. The terrestrial infrastructure presents itself as a preeminent component in this change. UAVs empowered with artificial intelligence are expected to become an integral component of future smart cities that provide seamless coverage for vehicles on highways with poor cellular infrastructure. We introduce UAVs cell-free network for providing coverage to vehicles entering a highway that is not covered; however, UAVs have limited energy resources and cannot serve the entire highway all the time. Furthermore, the deployed UAVs have insufficient knowledge about the environment; therefore, it is challenging to control a swarm of UAVs to achieve efficient communication coverage. To address these challenges, we formulate the trajectories decisions making as a Markov decision process where the system state space considers the vehicular network dynamics. Then, we leverage deep reinforcement learning to propose an approach for learning the optimal trajectories of the deployed UAVs to efficiently maximize the coverage, where we adopt Actor-Critic algorithm to learn the vehicular environment and its dynamics to handle the complex continuous action space. Finally, simulations results are provided to verify our findings.

In order to support the development of 5G technologies, researchers are actively engaged in addressing the challenges accompanying the emerging 5G applications. Unquestionably, an eminent technology, which is recently gaining significant research attention is mobile edge computing. Vehicular Edge Computing, an extension of cloud computing, brings capabilities to deploy data storage, computing and communication, as well as a host of support applications for emerging vehicular services (such as traffic control and safety, coordinated and automated driving, connected cars, intersection management) as well as emerging applications which require intensive computation and demand low-delay data processing, (e.g., video analytics with real-time interaction, image-aided navigation, content caching and sharing, natural language processing, etc.) to the edge closer to the vehicles, reducing response times and increasing reliability, therefore achieving the holistic vision of the tactile Internet. In this context, this paper considers a vehicular network with VEC capabilities deployed at road side units, RSUs, and addresses the problem of workload offloading as well as scheduling of computation tasks on the computing resources available at the edge. The challenge here is related to the high mobility of the vehicles and hence their short residence time within the coverage range of the RSU hosting the VEC. A joint problem considering the communication and computation resources, as well as the latency requirements of the workload is formulated and the scheduling is shown to be NP-Hard. Subsequently, efficient polynomial time solutions based on Lagrangian relaxation are derived and presented. We evaluate numerically the proposed methods and show their closeness to the optimal solutions obtained by a derived ILP.