Edge computing (EC) extends the ability of cloud computing to the network edge to support diverse resource-sensitive and performance-sensitive IoT applications. However, due to the limited capacity of edge servers (ESs) and the dynamic computing requirements, the system needs to dynamically update the task allocation policy according to real-time system states. Service migration is essential to ensure service continuity when implementing dynamic task allocation. Therefore, this article investigates the long-term dynamic task allocation and service migration (DTASM) problem in edge-cloud IoT systems where users’ computing requirements and mobility change over time. The DTASM problem is formulated to achieve the long-term performance of minimizing the load forwarded to the cloud while fulfilling the seamless migration constraint and the latency constraint at each time of implementing the DTASM decision. First, the DTASM problem is divided into two subproblems: 1) the user selection problem on each ES and 2) the system task allocation problem. Then, the DTASM problem is formulated as a Markov decision process (MDP) and an approach based on deep reinforcement learning (DRL) is proposed. To tackle the challenge of vast discrete action spaces for DTASM task allocation in the system with a mass of IoT users, a training architecture based on the twin-delayed deep deterministic policy gradient (DDPG) is employed. Meanwhile, each action is divided into a differentiable action for policy training and one mapped action for implementation in the IoT system. Simulation results demonstrate that the proposed DRL-based approach obtains the long-term optimal system performance compared to other benchmarks while satisfying seamless service migration.

Edge servers can collaborate to enhance service capability. However, cloud servers may be unable to execute centralized management due to unpredictable communications. In such systems, distributed task and resource management are vital but challenging due to heterogeneity and various restrictions. Therefore, this paper studies such edge systems and formulates the distributed joint task and computing resource allocation problem for maximizing the quality of experience (QoE). Given the restrictions on real-time state observations and resource management involving other facilities, we decompose it into sub-problems of distributed task allocation and computing resource allocation. After formulating the problem as a partially observed Markov decision process, we propose a two-step approach that depends on multi-agent (MA) deep reinforcement learning. First, each edge server performs a policy to allocate tasks for its associated users according to a partial observation. We employ the MA deep deterministic policy gradient to tackle vast spaces of discrete actions. Besides, we incorporate the action entropy of massive users' task allocation to enhance exploration. Then, we prove that the QoE-maximized computing resource allocation is a problem of maxing a sum of sigmoids, and we address it by sigmoidal programming. Simulation results reveal that the proposed approach dramatically improves the system QoE and reduces the average service latency. Besides, the proposed solution outperforms benchmarks in training and convergence.

By placing edge service functions in proximity to IoT facilities, edge computing can satisfy various IoT applications’ resource and latency requirements. Sensing-data-driven IoT applications are prevalent in IoT systems, and their task processing relies on sensing data from sensors. Therefore, to ensure the Quality of Service (QoS) of such applications in an edge-enabled IoT system, dedicated caching functions (CFs) are required to cache necessary sensing data. This article considers an edge-enabled IoT system and investigates the joint caching and computing service placement (JCCSP) problem for sensing-data-driven IoT applications. Then, deep reinforcement learning (DRL) is exploited to address the problem since it can adapt to a heterogeneous system with limited prior knowledge. In the proposed DRL-based approaches, a policy network based on the encoder–decoder model is constructed to address the issue of varying sizes of JCCSP states and actions caused by different numbers of CFs related to applications. Then, an on-policy REINFORCE-based method is adopted to train the policy network. After that an off-policy training method based on the twin-delayed (TD) deep deterministic policy gradient (DDPG) is proposed to enhance the training efficiency and experience utilization. In the proposed DDPG-based method, a weight-averaged twin- $Q$ -delayed (WATQD) algorithm is introduced to reduce the bias of $Q$ -value estimation. Simulation results show that our proposed DRL-based JCCSP approaches can achieve converged performance that is significantly superior to benchmarks. Moreover, compared with the original TD method, the proposed WATQD method can significantly improve the training stability.

Edge computing brings powerful computing ability to the proximity of IoT devices to guarantee latency constraints, making it one essential technology for supporting intelligent applications in future Internet of Things (IoT). Collaboration among edge computing servers (ECSs) with limited resources is an efficient solution to enhance the capability of edge network, and placement of ECSs and service functions (SFs) impose significant influences on system performance. This paper explores the collaboration among ECSs by considering the simultaneous and heterogeneous consumption of different computing resources. The service deployment and application assignment in regional edge computing enabled IoT (EdgeIoT) are investigated. A collaborative service deployment and application assignment (ColSDA) algorithm is proposed to render the final edge service deployment strategy, including the placement of ECSs and SFs as well as the assignment of applications to ECSs. In ColSDA, the minimum number of ECSs to be placed is obtained by the proposed minimum resource ration increase (MinRI) algorithm. Computing loads are then balanced by the load-balancing reassignment (LBRA) algorithm. After placing ECSs, a search and swap (SeSw) algorithm is proposed to further increase the number of tasks processed by locally deployed ECSs. Simulation results demonstrate that the number of required ECSs under the premise of guaranteeing the quality of service (QoS) can be significantly reduced by establishing collaboration among ECSs. Besides, the proposed ColSDA algorithm can provide the service deployment and application assignment strategy for a given region EdgeIoT as expected.