When using blockchain in mobile systems, computation intensive mining tasks pose great challenges to the processing capabilities of mobile miner equipment. Mobile edge computing (MEC) is an effective solution to alleviating the problem via task offloading. In the mining process, miners compete for rewards through puzzle solving, where only the miner that first completes the process will be rewarded. Thus, miners may wish to pay higher price and use more communication resources in task offloading and more computation resources in task processing for latency reduction. However, there are risks for the miners not profiting from consuming more resources or paying a higher price, so miners are rational in blockchain systems. In order to maximize the rational total profit of all miners, we use an asynchronous advantage actor-critic (A3C) deep reinforcement learning algorithm to obtain the resource pricing and allocation, considering the stochastic properties of wireless channels, and the prospect theory is employed to strike a good balance between risks and rewards. Numerical results show that our proposed A3C based joint optimization algorithm converges fast and outperforms the baseline algorithms in terms of the total reward.

The Internet of Things (IoT) has already emerged as one of the most popular directions in today’s information and communication technology (ICT) domain. With its advancement over different application areas, such as smart home, smart healthcare, industry 4.0, etc., a huge amount of data has been generated by billions of IoT devices, which aggravates the shortcomings of the network layer (IP)-based networks, such as limited expressiveness of IP addressing, inefficient support for mobility, and in-network caching. Building IoT on top of information-centric networking (ICN) is believed to be a promising solution to tackle the above challenge, especially the in-network caching of ICN can significantly benefit IoT in terms of reducing data and saving IoT devices’ energy. However, caching IoT data is more challenging than caching traditional Internet content, e.g., video, because IoT data are usually valid within a certain period of time, and IoT devices are typically constrained with battery. Hence, in this survey, we first review the current implementation proposals of ICN-based IoT (ICN-IoT). Next, we present the conventional caching decision policies and replacement policies which could be adopted to mitigate the aforementioned challenges, e.g., reducing IoT traffic, saving energy, and reducing data retrieval latency. Further, since leveraging machine learning (ML) techniques have the potential to further improve the caching efficiency by dealing with uncertainties, e.g., predicting unknown information, adaptively interacting with the environment, we also demonstrate the recently proposed ML-based caching schemes for ICN-IoT. In addition, we outline the open research issues and point out the future opportunities of caching in ICN-IoT.

With the rapid development of automotive industry and telecommunication technologies, live streaming services in the Internet of vehicles (IoV) play an even more crucial role in vehicular infotainment systems. However, it is a big challenge to provide a high quality, low latency, and low bitrate variance live streaming service for vehicles due to the dynamic properties of wireless resources and channels of IoV. To solve this challenge, we propose a novel live video transcoding and streaming scheme that maximizes the video bitrate and decreases time-delays and bitrate variations in vehicle fog computing (VFC)-enabled IoV, by jointly optimizing vehicle scheduling, bitrate selection, and computational/spectrum resource allocation. This joint optimization problem is modeled as a Markov decision process (MDP), considering time-varying characteristics of the available resources and wireless channels of IoV. A soft actor-critic deep reinforcement learning (DRL) algorithm that is based on the maximum entropy framework, is subsequently utilized to solve the above MDP. Extensive simulation results based on the dataset of the real world show that compared to other baseline algorithms, the proposed scheme can effectively improve video quality while decreasing latency and bitrate variations, and access excellent performance in terms of learning speed and stability.

Unmanned aerial vehicles (UAVs) are anticipated to be integrated into the next generation wireless networks as new aerial mobile nodes, which can provide various live streaming applications such as surveillance, reconnaissance, etc. For such applications, due to the dynamic characteristics of traffic and wireless channels, how to guarantee the quality of service (QoS) is a challenging task. In this paper, with recent advances in scalable video coding (SVC), we study energy-efficient secure video streaming in rotary-wing UAV-enabled wireless networks. By jointly optimizing video levels selection, power allocation, and the UAV’s trajectory, we intend to maximize the long-term energy efficiency that is defined as the ratio of video quality to power consumption. Meanwhile, secrecy timeout probability is considered as a constraint cost to guarantee time delays requirements in a long run perspective. Our problem is modeled as a constrained Markov decision process (CMDP) and solved by safe deep Q-learning network (safe-DQN), where a safe policies set induced by constructing a Lyapunov function is dynamically adjusted to satisfy the constraint conditions of the CMDP. Extensive simulation results with different system parameters show the effectiveness of the proposed algorithm compared with other existing reinforcement learning algorithms.

With the advances in hand-held devices (smart-phones and tablets, etc.) and high speed wireless networks, users have an explosive growth demand for live streaming service. Due to the diversity of user equipments (UEs), the live streaming has to be transcoded into different versions. However, transcoding is a computationally expensive and time consuming process. Due to the dynamic characteristics of wireless network environment, providing high quality and low latency live videos for UEs is a big challenge. This study investigates user scheduling, transcoding decision, computational and wireless spectrum resources allocation problem in edge-clouds aided heterogeneous networks (Het-Nets). The research focuses on improving UEs' quality of service (QoS) for live-streaming services, which includes quality of video and latency requirements. Different from existing literature, to approach the real wireless environment, the available computa- tional and wireless spectrum resources are modeled as random processes in the work. Considering dynamic characteristics of wireless networks and available resources, the above problem is modeled as a Markov decision process (MDP). Since the action space of the MDP is multi-dimensional continuous variables mixed with discrete variables, it is difficult to solve this problem by traditional learning algorithms. Therefore, an enhanced actor-critic algorithm is proposed to resolve the problem, in which both the actor part and the critic part employ eligibility traces. Extensive simulation results with different system parameters show the effectiveness of the proposed algorithm.