18 June 2020 in IEEE Internet of Things Journal
With the rapid development of automotive industry and telecommunication technologies, live streaming services in the Internet of vehicles (IoV) play an even more crucial role in vehicular infotainment systems. However, it is a big challenge to provide a high quality, low latency, and low bitrate variance live streaming service for vehicles due to the dynamic properties of wireless resources and channels of IoV. To solve this challenge, we propose a novel live video transcoding and streaming scheme that maximizes the video bitrate and decreases time-delays and bitrate variations in vehicle fog computing (VFC)-enabled IoV, by jointly optimizing vehicle scheduling, bitrate selection, and computational/spectrum resource allocation. This joint optimization problem is modeled as a Markov decision process (MDP), considering time-varying characteristics of the available resources and wireless channels of IoV. A soft actor-critic deep reinforcement learning (DRL) algorithm that is based on the maximum entropy framework, is subsequently utilized to solve the above MDP. Extensive simulation results based on the dataset of the real world show that compared to other baseline algorithms, the proposed scheme can effectively improve video quality while decreasing latency and bitrate variations, and access excellent performance in terms of learning speed and stability.
7 July 2021 in IEEE Transactions on Green Communications and Networking
Unmanned aerial vehicles (UAVs) are anticipated to be integrated into the next generation wireless networks as new aerial mobile nodes, which can provide various live streaming applications such as surveillance, reconnaissance, etc. For such applications, due to the dynamic characteristics of traffic and wireless channels, how to guarantee the quality of service (QoS) is a challenging task. In this paper, with recent advances in scalable video coding (SVC), we study energy-efficient secure video streaming in rotary-wing UAV-enabled wireless networks. By jointly optimizing video levels selection, power allocation, and the UAV’s trajectory, we intend to maximize the long-term energy efficiency that is defined as the ratio of video quality to power consumption. Meanwhile, secrecy timeout probability is considered as a constraint cost to guarantee time delays requirements in a long run perspective. Our problem is modeled as a constrained Markov decision process (CMDP) and solved by safe deep Q-learning network (safe-DQN), where a safe policies set induced by constructing a Lyapunov function is dynamically adjusted to satisfy the constraint conditions of the CMDP. Extensive simulation results with different system parameters show the effectiveness of the proposed algorithm compared with other existing reinforcement learning algorithms.
20 September 2019 in IEEE Transactions on Vehicular Technology
With the advances in hand-held devices (smart-phones and tablets, etc.) and high speed wireless networks, users have an explosive growth demand for live streaming service. Due to the diversity of user equipments (UEs), the live streaming has to be transcoded into different versions. However, transcoding is a computationally expensive and time consuming process. Due to the dynamic characteristics of wireless network environment, providing high quality and low latency live videos for UEs is a big challenge. This study investigates user scheduling, transcoding decision, computational and wireless spectrum resources allocation problem in edge-clouds aided heterogeneous networks (Het-Nets). The research focuses on improving UEs' quality of service (QoS) for live-streaming services, which includes quality of video and latency requirements. Different from existing literature, to approach the real wireless environment, the available computa- tional and wireless spectrum resources are modeled as random processes in the work. Considering dynamic characteristics of wireless networks and available resources, the above problem is modeled as a Markov decision process (MDP). Since the action space of the MDP is multi-dimensional continuous variables mixed with discrete variables, it is difficult to solve this problem by traditional learning algorithms. Therefore, an enhanced actor-critic algorithm is proposed to resolve the problem, in which both the actor part and the critic part employ eligibility traces. Extensive simulation results with different system parameters show the effectiveness of the proposed algorithm.
5 June 2023 in IEEE Internet of Things Journal
Live traffic video is vitally important for vehicles in future intelligent transport systems (ITSs). Due to the limitation of onboard sensors, vehicles may not be able to obtain a full view of the traffic situations which endangers safety for autonomous driving vehicles. In this article, we propose a traffic video multicasting scheme by using video splitting and group splitting techniques for unmanned aerial vehicles (UAVs)-assisted ITS, in which UAVs are considered as the eyes in the sky to capture real-time traffic videos. We aim to maximize the long-term video quality received by vehicles by jointly optimizing vehicle grouping and spectrum allocation. Considering the interactions among UAVs, the above optimization problem is formulated as a multiagent coordination problem in the form of a Markov game (MG). The MG is subsequently solved by leveraging a state-of-the-art multiagent deep reinforcement learning (MADRL) algorithm, namely, multiactor attention critic (MAAC), in which an attention mechanism is utilized to pay attention to other agents to make the learning process more effective and scalable. Extensive simulation results show that the MAAC-based algorithm has better performance in terms of video quality and spectrum efficiency compared with the baseline methods.
3 October 2023 in IEEE Transactions on Network Science and Engineering
Thanks to smart manufacturing and artificial intelligence technologies, unmanned aerial vehicles (UAVs) are envisioned to play a critical role in future Internet of things (IoT) networks to execute data collection tasks. In this paper, we leverage age of information (AoI) to measure the freshness of data packets received by the UAV from IoT sensors. Considering the heterogeneity of IoT devices, we aim to minimize the weighted sum AoI by jointly optimizing the UAV's trajectory and IoT devices association in UAV-assisted IoT networks, where the UAV's cumulative propulsion energy cost is limited by the battery capacity. Since the optimization object is confined by a set of short-term constraints and a long-term constraint, this problem is modeled as a constrained Markov decision process (CMDP). We leverage safe actor-critic (Safe-AC) to solve the CMDP. To satisfy the mixed constraints, the safe policy set of Safe-AC is induced by a Lyapunov function, thereafter, a policy distillation technology is leveraged to search the optimal policy. Experimental results indicate that our proposed scheme can strictly satisfy the propulsion energy cost budget requirement at the expense of around $2\%$ loss of the reward compared to baseline methods.
27 March 2024 in IEEE Internet of Things Journal
Unmanned aerial vehicles (UAVs) equipped with high definition (HD) cameras, intelligent sensors, computing, and communication modules can be deployed to execute crowdsensing tasks by leveraging federated learning (FL), e.g., air quality perception and ground target detection. FL can reduce transmission stress and protect data privacy when training models, which is suitable for resource constrained Internet of UAVs. Nevertheless, the incentive issues about information asymmetry and bounded rationality impede the applications of FL-enabled Internet of UAVs. The existing FL incentive approaches focus on the risk-free condition, where task publishers are capable of making decisions with complete rationality by utilizing expected utility theory. In fact, task publishers under risk conditions are often bounded rational, whose risk-awareness makes the utility models more sophisticated. To overcome the above problems, we present a prospect theory (PT)-based incentive mechanism for FL-enabled Internet of UAVs. We first leverage PT to model the task publisher’s risk-awareness behavior and construct the subjective utility model. Thereafter, we utilize the framing effect of PT to design the optimal contract to maximize the subjective utility. Simulation results demonstrate that, compared with the baseline method, the proposed incentive mechanism has better performance.