In this paper, we establish a two-timescale framework for the joint service deployment and task scheduling problem in satellite edge computing networks.We aim to optimize the computing performance of networks with diverse quality-of-service (QoS) guarantees for computing tasks. Specifically, to capture the small-timescale network dynamics and task randomness, we formulate the task scheduling problem as a constrained Markov decision process (CMDP) to minimize the energy consumption, load imbalance and packet loss of networks while ensuring the long-term delay. The Lyapunov technique is employed to deal with the delay constraints. A soft actor-critic (SAC)-based deep reinforcement learning (DRL) framework is designed to learn the stationary scheduling policy. We further explore the significant impact of deploying diverse services on the performance of task scheduling in satellite edge computing. Considering that frequent deployment of services will incur huge deployment overhead, we optimize the service deployment on a larger timescale. The optimization problem is modeled as an integer programming problem to improve the service capability of networks and reduce service deployment costs. A heuristic-based atomic orbital search (AOS) approach is proposed to obtain the superior policy with low complexity. Due to the correlation between the problems of two timescales, a hierarchical solution is constructed to iteratively find the excellent solution. Finally, extensive simulations are conducted to validate the effectiveness and superiority of the proposed scheme.

By delegating the infrastructure management, such as provisioning or scaling to third-party providers, serverless edge computing has recently been widely adopted in several applications, especially Internet of Things (IoT) applications. Task scheduling is a critical issue in serverless edge computing as it significantly impacts the quality of user experience. In contrast to the centralized scheduling in the cloud center, serverless edge task scheduling is more challenging due to the heterogeneous and resource-constrained nature of edge resources. This article aims to study the distributed task scheduling for the IoT in serverless edge computing networks, in which heterogeneous serverless edge computing nodes are rational individuals with interests to optimize their own scheduling utility while the nodes only have access to local observations. The task scheduling competition process is formulated as a partially observable stochastic game (POSG) to enable serverless edge computing nodes to noncooperatively schedule tasks and allocate computing resources depending on their locally observed system state, which takes into account the associated task generation state, data queue state, communication channel state, and previous computing resource allocation state. To solve the proposed POSG and deal with the partial observability, a multiagent task scheduling algorithm based on the dueling double deep recurrent $Q$ -network (D3RQN) method is developed to approximate the optimal task scheduling and resource allocation solution. Finally, extensive simulation experiments are conducted to validate the effectiveness and superiority of the proposed scheme.

Recently, fog computing has emerged as a prospective technique to provide pervasive and agile computation services for Internet of things (IoT) devices and support advanced applications. Introducing the energy harvesting (EH) technique into the fog computing system can extend the battery lifetime and provide a higher quality of experiences (QoE) for IoT devices. In EH enabled IoT fog system, computation offloading is an important issue and has attracted much attention. In most existing works, it is assumed that the IoT device is fully aware of the system state. However, in practical offloading problems, the IoT device may not be able to obtain accurate system state information, and only have a partial observation of the environment. Therefore, in this paper, we investigate the decentralized partially observable offloading problem in the EH enabled IoT fog system, in which multiple IoT devices cooperate to maximize the network performance while meeting their QoE requirements. We formulate the optimization problem as a decentralized partially observable Markov decision process (Dec-POMDP) in which each IoT device makes the task offloading decisions according to its local observation of the environment. The Lagrangian approach and the policy gradient method are adopted to find the optimal solution for the proposed problem. Due to the high complexity of solving the Dec-POMDP, a learning-based decentralized offloading algorithm with low complexity is presented to find the approximate optimal solution. Finally, extensive experimental evaluation and comparison are carried out to show the effectiveness of the proposed scheme.

Serverless edge computing is seen as a promising enabler to execute differentiated Industrial Internet of Things (IIoT) applications without managing the underlying servers and clusters. In IIoT serverless edge computing, IIoT workflow scheduling for cloud-edge collaborative processing is closely related to the service quality of users. However, serverless functions decomposed by IIoT applications are limited in their deployment at the edge due to the resource-constrained nature of edge infrastructures. In addition, the scheduling of complex IIoT applications supported by serverless computing is more challenging. Therefore, considering the limited function deployment and the complex dependencies of serverless workflows, we model the workflow application as directed acyclic graph and formulate the scheduling problem as a multiobjective optimization problem. A dueling double deep Q-network-based solution is proposed to make scheduling decisions under dynamically changing systems. Extensive simulation experiments are conducted to validate the superiority of the proposed scheme.

Edge intelligence is emerging as a new interdiscipline to push learning intelligence from remote centers to the edge of the network. However, with its widespread deployment, new challenges arise in terms of training efficiency and service of quality (QoS). Massive repetitive model training is ubiquitous due to the inevitable needs of users for the same types of data and training results. Additionally, a smaller volume of data samples will cause the over-fitting of models. To address these issues, driven by the Internet of intelligence, this paper proposes a distributed edge intelligence sharing scheme, which allows distributed edge nodes to quickly and economically improve learning performance by sharing their learned intelligence. Considering the time-varying edge network states including data collection states, computing and communication states, and node reputation states, the distributed intelligence sharing is formulated as a multi-agent Markov decision process (MDP). Then, a novel collective deep reinforcement learning (CDRL) algorithm is designed to obtain the optimal intelligence sharing policy, which consists of local soft actor-critic (SAC) learning at each edge node and collective learning between different edge nodes. Simulation results indicate our proposal outperforms the benchmark schemes in terms of learning efficiency and intelligence sharing efficiency.

Viewport prediction and prefetch have an important influence on VR video streaming performance. This work proposes a novel federated learning-based viewport prediction model training algorithm, ComPer-FedAvg. The proposed algorithm leverages a VR video’s common viewing pattern and users’ personal viewing patterns to train the prediction model in a distributed and privacy-preserving manner. Further, considering the VR video viewport prediction accuracy, a stochastic game is formulated to solve the VR streaming network’s communication resource allocation problem, where limited communication resource blocks are auctioned to users to achieve the optimal overall VR viewing experience. For each user, the auction is decomposed into two disjoint subproblems, namely, the optimal number of data rate requesting and true value claiming (bidding). The optimal true value claiming has been analytically proved to be equal to the VR viewing reward with given data rate. Due to the lack of global information when users request data rate, we reformulate users’ data rate requesting problem as a POMDP problem. A novel deep reinforcement learning algorithm is adopted to solve the problem. Evaluation and simulation results show the proposed viewport prediction and VR streaming schemes outperform conventional solutions in terms of prediction accuracy and VR viewing experience.