With the rapid development of wireless communication technologies, the emerging multimedia applications make mobile Internet traffic grow explosively while putting forward higher service requirements for the next-generation wireless networks. Therefore, how to achieve low-latency content transmission by effectively allocating heterogeneous network resources to improve the network quality of service and end-user quality of experience is a key issue to be solved urgently in the current Internet. In this article, we propose a deep reinforcement learning (DRL)-based resource allocation scheme to improve content distribution in a layered fog radio access network (FRAN). We formulate the optimal resource allocation problem as a minimal delay model, where in-network caching is deployed and the same content requests from mobile users can be aggregated in the queue of each base station. To cope with the increasing user requests and overcome capacity constraints of the FRAN, moreover, a cloud–edge cooperation offloading scheme is utilized in our model, where the integrated allocation of caching, computing, and communication resources and joint optimization between in-network caching and routing are considered to promote resource utilization and content delivery. In our solution, a new DRL policy is designed to make cross-layer cooperative caching and routing decisions for the arriving content requests according to request history information and available network resources in the system. Simulation results demonstrate that our proposed model can performs much better than the existing cloud–edge cooperation schemes in the FRAN.

With the proliferation of mobile devices (e.g., vehicles and smartphones), rich media content services from massive users lead to high network resource consumption and energy usage. How to effectively allocate heterogeneous network resources and achieve green content delivery is a major challenge to be addressed in urgency, especially when vehicular users are involved and the spatiotemporal distribution of the content requests can change drastically. In this paper, we propose a new deep reinforcement learning (DRL)-aided task offloading and resource allocation scheme, named TORA-DRL, to minimize power consumption in cloud-edge cooperation environments, where in-network caching and request aggregation are incorporated to reduce replicated transmissions of network contents. Based on the history of content requests and available network resources in the system, TORA-DRL jointly optimizes the decisions about task offloading, as well as computing, caching and communication resource allocation, adapting to the changes in network states and user requirements. Simulation results demonstrate that the proposed TORA-DRL solution converges fast and has much higher energy efficiency than the existing popular cloud-edge cooperation schemes under different network environments.

With the advancement of mobile communication technology, there has been a marked increase in the demand for personalized and ubiquitous Internet of Things (IoT) services, raising the expectations for network Quality of Service (QoS) and Quality of Experience (QoE). Existing popularity-prediction-based content caching policies improve QoS and QoE by precaching contents at the network edge, but jointly optimizing multiple network metrics remains a challenge. To address this challenge, we propose a many-objective optimization-based popularity prediction for cooperative caching (MaOPPC-Caching) framework for cloud–edge–end collaborative IoT networks. This framework simultaneously optimizes prediction accuracy, delay, offloaded traffic, and load balance. We integrate three prediction algorithms to forecast content popularity and present a horizontal and vertical collaborative caching decision strategy to generate caching forms based on the predicted results. Then, the many-objective evolutionary algorithm (MaOEA) is employed to optimize the combined proportions to take full advantage of hidden preferences and popularity characteristics of both users and items. To promote the convergence of the framework, we present a knowledge mining-based MaOEA (KMaOEA) to incorporate knowledge mining into the optimization process. Simulation results show that the proposed MaOPPC-Caching framework outperforms existing prediction algorithms in terms of four evaluation indicators. Furthermore, KMaOEA shows a significant advantage over NSGA-III in load balance, as indicated by a Mann–Whitney rank sum test with a $p$ -value of 0.040.

To tackle a challenging energy efficiency problem caused by the growing mobile Internet traffic, this paper proposes a deep reinforcement learning (DRL)-based green content task offloading scheme in cloud-edge-end cooperation networks. Specifically, we formulate the problem as a power minimization model, where requests arriving at a node for the same content can be aggregated in its queue and in-network caching is widely deployed in heterogeneous environments. A novel DRL algorithm is designed to minimize the power consumption by making collaborative caching and task offloading decisions in each slot on the basis of content request information in previous slots and current network state. Numerical results show that our proposed content task offloading model achieves better power efficiency than the existing popular counterparts in cloud-edge-end collaboration networks, and fast converges to the stable state.

Complex dynamic services and heterogeneous network environments make the asymmetrical control a curial issue to handle on the Internet. With the advent of the Internet of Things (IoT) and the fifth generation (5G), the emerging network applications lead to the explosive growth of mobile traffic while bringing forward more challenging service requirements to future radio access networks. Therefore, how to effectively allocate limited heterogeneous network resources to improve content delivery for massive application services to ensure network quality of service (QoS) becomes particularly urgent in heterogeneous network environments. To cope with the explosive mobile traffic caused by emerging Internet services, this paper designs an intelligent optimization strategy based on deep reinforcement learning (DRL) for resource allocation in heterogeneous cloud-edge-end collaboration environments. Meanwhile, the asymmetrical control problem caused by complex dynamic services and heterogeneous network environments is discussed and overcome by distributed cooperation among cloud-edge-end nodes in the system. Specifically, the multi-layer heterogeneous resource allocation problem is formulated as a maximal traffic offloading model, where content caching and request aggregation mechanisms are utilized. A novel DRL policy is proposed to improve content distribution by making cache replacement and task scheduling for arriving content requests in accordance with the information about users’ history requests, in-network cache capacity, available link bandwidth and topology structure. The performance of our proposed solution and its similar counterparts are analyzed in different network conditions.

A simultaneously transmitting and reflecting surface (STARS) enabled edge caching system is proposed for reducing backhaul traffic and ensuring the quality of service. A novel Caching-at-STARS structure, where a dedicated smart controller and cache memory are installed at the STARS, is proposed to satisfy user demands with fewer hops and desired channel conditions. Then, a joint caching replacement and information-centric hybrid beamforming optimization problem is formulated for minimizing the network power consumption. As long-term decision processes, the optimization problems based on independent and coupled phase-shift models of Caching-at-STARS contain both continuous and discrete decision variables, and are suitable for solving with deep reinforcement learning (DRL) algorithm. For the independent phase-shift Caching-at-STARS model, we develop a frequency-aware based twin delayed deep deterministic policy gradient (FA-TD3) algorithm that leverages user historical request information to serialize high-dimensional caching replacement decision variables. For the coupled phase-shift Caching-at-STARS model, we conceive a cooperative TD3 & deep-Q network (TD3-DQN) algorithm comprised of FA-TD3 and DQN agents to decide on continuous and discrete variables respectively by observing the network external and internal environment. The numerical results demonstrate that: 1) The Caching-at-STARS-enabled edge caching system has advantages over traditional edge caching, especially in scenarios where Zipf skewness factors or cache capacity is large; 2) Caching-at-STARS outperforms the RIS-assisted edge caching systems; 3) The proposed FA-TD3 and cooperative TD3-DQN algorithms are superior in reducing network power consumption than conventional TD3.