Edge caching is an emerging technology for addressing massive content access in mobile networks to support rapidly growing Internet of Things (IoT) services and applications. However, most current optimization-based methods lack a self-adaptive ability in dynamic environments. To tackle these challenges, current learning-based approaches are generally proposed in a centralized way. However, network resources may be overconsumed during the training and data transmission process. To address the complex and dynamic control issues, we propose a FederAted Deep reinforcement learning-based cooperative Edge caching (FADE) framework. FADE enables base stations (BSs) to cooperatively learn a shared predictive model by considering the first-round training parameters of the BSs as the initial input of the local training, and then uploads near-optimal local parameters to the BSs to participate in the next round of global training. Furthermore, we prove the expectation convergence of FADE. Trace-driven simulation results demonstrate the effectiveness of the proposed FADE framework on reducing the performance loss and average delay, offloading backhaul traffic and improving hit rate.

Mobile-edge computing (MEC) has emerged as a promising computing paradigm in the 5G architecture, which can empower user equipments (UEs) with computation and energy resources offered by migrating workloads from UEs to the nearby MEC servers. Although the issues of computation offloading and resource allocation in MEC have been studied with different optimization objectives, they mainly focus on facilitating the performance in the quasistatic system, and seldomly consider time-varying system conditions in the time domain. In this article, we investigate the joint optimization of computation offloading and resource allocation in a dynamic multiuser MEC system. Our objective is to minimize the energy consumption of the entire MEC system, by considering the delay constraint as well as the uncertain resource requirements of heterogeneous computation tasks. We formulate the problem as a mixed-integer nonlinear programming (MINLP) problem, and propose a value iteration-based reinforcement learning (RL) method, named $Q$ -Learning, to determine the joint policy of computation offloading and resource allocation. To avoid the curse of dimensionality, we further propose a double deep $Q$ network (DDQN)-based method, which can efficiently approximate the value function of $Q$ -learning. The simulation results demonstrate that the proposed methods significantly outperform other baseline methods in different scenarios, except the exhaustion method. Especially, the proposed DDQN-based method achieves very close performance with the exhaustion method, and can significantly reduce the average of 20%, 35%, and 53% energy consumption compared with offloading decision, local first method, and offloading first method, respectively, when the number of UEs is 5.

In order to meet the growing demands for multimedia service access and release the pressure of the core network, edge caching and device-to-device (D2D) communication have been regarded as two promising techniques in next generation mobile networks and beyond. However, most existing related studies lack consideration of effective cooperation and adaptability to the dynamic network environments. In this paper, based on the flexible trilateral cooperation among user equipment, edge base stations and a cloud server, we propose a D2D-assisted heterogeneous collaborative edge caching framework by jointly optimizing the node selection and cache replacement in mobile networks. We formulate the joint optimization problem as a Markov decision process, and use a deep Q-learning network to solve the long-term mixed integer linear programming problem. We further design an attention-weighted federated deep reinforcement learning (AWFDRL) model that uses federated learning to improve the training efficiency of the Q-learning network by considering the limited computing and storage capacity, and incorporates an attention mechanism to optimize the aggregation weights to avoid the imbalance of local model quality. We prove the convergence of the corresponding algorithm, and present simulation results to show the effectiveness of the proposed AWFDRL framework in reducing average delay of content access, improving hit rate and offloading traffic.

Recently, due to the increasing popularity of enjoying various multimedia services on mobile devices (e.g., smartphones, ipads and electronic tablets), the generated mobile data traffic has been explosively growing and has become a serve burden on mobile network operators. To address such a serious challenge in mobile networks, an effective approach is to manage data traffic by using complementary technologies (e.g., small cell network, WiFi network, and so on) to achieve mobile data offloading. In this paper, we discuss the recent advances in the techniques of mobile data offloading. Particularly, based on the initiator diversity of data offloading, we classify the existing mobile data offloading technologies into four categories, i.e., data offloading through small cell networks, data offloading through WiFi networks, data offloading through opportunistic mobile networks and data offloading through heterogeneous networks. Besides, we show a detailed taxonomy of the related mobile data offloading technologies by discussing the pros and cons for various offloading technologies for different problems in mobile networks. Finally, we outline some opening research issues and challenges, which can provide guidelines for future research work.

The explosive growth of content requests from mobile users is stretching the capability of current mobile networking technologies to satisfy users’ demands with acceptable quality of service. An effective approach to address this challenge, which has not yet been thoroughly studied, is to offload network traffic by caching popular contents at the edges (e.g., mobile devices and base stations) of mobile networks, thus reducing the massive duplication of content downloads. In this paper, we address the system modeling, large-scale optimization and framework design of hierarchical edge caching in device-to-device (D2D) aided mobile networks. In particular, taking into account the analysis of social behavior and preference of mobile users, heterogeneous cache sizes, and the derived system topology, we investigate the maximum capacity of the network infrastructure in terms of offloading network traffic, reducing system costs and supporting content requests from mobile users locally. Our proposed framework has a low complexity, and can be applied in practical engineering implementation. Trace-based simulation results demonstrate the effectiveness of the proposed framework.

To deal with the explosive growth in multimedia service requests in mobile networks, caching contents at the cells (base stations) is regarded as an effective emerging technique to reduce the duplicated transmissions of content downloads, while heterogeneous networks (HetNets) are regarded as an effective technique to increase the network capacity. Yet, the combination of content caching and HetNets for future networks (i.e., 5G) is still not well explored. In this paper, we propose an efficient collaborative multi-tier caching framework in Het-Nets. In particular, based on patterns of user requests, link capacities, heterogenous cache sizes, and the derived system topology, we focus on exploring the maximum capacity of the network infrastructure so as to offload the network traffic and support users' content requests locally. Due to the NP-hardness of the complex multi-tier caching problem, we approximately decompose it into some subproblems that focus on the caching cooperation at different tiers by utilizing the derived system topology. Our proposed framework is low-complexity and distributed, and can be used for practical engineering implementation. Trace-based simulation results demonstrate the effectiveness of the proposed framework.