CCTV-based surveillance using unmanned aerial vehicles (UAVs) is considered a key technology for security in smart city environments.This article creates a case where the UAVs with CCTV-cameras fly over the city area for flexible and reliable surveillance services. UAVs should be deployed to cover a large area while minimizing overlapping and shadow areas for a reliable surveillance system. However, the operation of UAVs is subject to high uncertainty, necessitating autonomous recovery systems. This article develops a multiagent deep reinforcement learning-based management scheme for reliable industry surveillance in smart city applications. The core idea this article employs is autonomously replenishing the UAV's deficient network requirements with communications. Via intensive simulations, our proposed algorithm outperforms the state-of-the-art algorithms in terms of surveillance coverage, user support capability, and computational costs.

State-of-the-art drone technologies have severe flight time limitations due to weight constraints, which inevitably lead to a relatively small amount of available energy. Therefore, frequent battery replacement or recharging is necessary in applications such as delivery, exploration, or support to the wireless infrastructure. Mobile charging stations (i.e., mobile stations with charging equipment) for outdoor ad-hoc battery charging is one of the feasible solutions to address this issue. However, the ability of these platforms to charge the drones is limited in terms of the number and charging time. This paper designs an auctionbased mechanism to control the charging schedule in multidrone setting. In this paper, charging time slots are auctioned, and their assignment is determined by a bidding process. The main challenge in developing this framework is the lack of prior knowledge on the distribution of the number of drones participating in the auction. Based on optimal second-priceauction, the proposed formulation, then, relies on deep learning algorithms to learn such distribution online. Numerical results from extensive simulations show that the proposed deep learningbased approach provides effective battery charging control in multi-drone scenarios.

This paper proposes a cloud-assisted joint charging scheduling and energy management framework for unmanned aerial vehicle (UAV) networks. For charging the UAVs those are extremely power hungry, charging towers are considered for plug-and-play charging during run-time operations. The charging towers should be cost-effective, thus it is equipped with photovoltaic power generation and energy storage systems functionalities. Furthermore, the towers should be cooperative for more cost-effectiveness by intelligent energy sharing. Based on the needs and setting, this paper proposes 1) charging scheduling between UAVs and towers and 2) cooperative energy managements among towers. For charging scheduling, the UAVs and towers should be scheduled for maximizing charging energy amounts and the scheduled pairs should determine charging energy allocation amounts. Here, two decisions are correlated, i.e. , it is a non-convex problem. We re-formulate the non-convex to convex for guaranteeing optimal solutions. Lastly, the cooperative energy sharing among towers is designed and implemented with multi-agent deep reinforcement learning and then intelligent energy sharing can be realized. We can observe that the two methods are related and it should be managed, coordinated, and harmonized by a centralized orchestration manager under the consideration of fairness, energy-efficiency, and cost-effectiveness. Our data-intensive performance evaluation verifies that our proposed framework achieves desired performance.

This article aims to provide a hierarchical reinforcement learning (RL)-based solution to the automated drug infusion field. The learning policy is divided into the tasks of: 1) learning trajectory generative model and 2) planning policy model. The proposed deep infusion assistant policy gradient (DIAPG) model draws inspiration from adversarial autoencoders (AAEs) and learns latent representations of hypnotic depth trajectories. Given the trajectories drawn from the generative model, the planning policy infers a dose of propofol for stable sedation of a patient under total intravenous anesthesia (TIVA) using propofol and remifentanil. Through extensive evaluation, the DIAPG model can effectively stabilize bispectral index (BIS) and effect site concentration given a potentially time-varying target sequence. The proposed DIAPG shows an increased performance of 530% and 15% when a human expert and a standard reinforcement algorithm are used to infuse drugs, respectively.

Total intravenous anesthesia using propofol and remifentanil is widely used. However, a rapid elimination property of the control of the drug infusion rate leads to problems. Recently, it has been shown that machine learning algorithms are able to more accurately predict drug effects than traditional prediction models. Based on this trend, this paper proposes a machine learning method that can simulate the transition dynamics of patient data during anesthesia to conFigure the interactive environment. In addition, we propose deep reinforcement learning with behavior cloning-based initialization to train the propofol and remifentanil control policies. As shown in the dataintensive performance evaluation, the proposed method achieves a desirable performance in terms of average bispectral index and blood pressure during the surgery simulation.