In the transportation industry, task offloading services of edge-enabled Internet of Vehicles (IoV) are expected to provide vehicles with the better Quality of Experience (QoE). However, the various status of diverse edge servers and vehicles, as well as varying vehicular offloading modes, make a challenge of task offloading service. Therefore, to enhance the satisfaction of QoE, we first introduce a novel QoE model. Specifically, the emerging QoE model restricted by the energy consumption: 1) intelligent vehicles equipped with caching spaces and computing units may work as carriers; 2) various computational and caching capacities of edge servers can empower the offloading; and 3) unpredictable routings of the vehicles and edge servers can lead to diverse information transmission. We then propose an improved deep reinforcement learning (DRL) algorithm named PS-DDPG with the prioritized experience replay (PER) and the stochastic weight averaging (SWA) mechanisms based on deep deterministic policy gradients (DDPG) to seek an optimal offloading mode, saving energy consumption. Specifically, the PER scheme is proposed to enhance the availability of the experience replay buffer, thus accelerating the training. Moreover, reducing the noise in the training process and thus stabilizing the rewards, the SWA scheme is introduced to average weights. Extensive experiments certify the better performance, i.e., stability and convergence, of our PS-DDPG algorithm compared to existing work. Moreover, the experiments indicate that the QoE value can be improved by the proposed algorithm.

In the current industry world, the industrial robot has emerged as a critical device to make the manufacturing process more efficient through automation. However, abnormal operation of industrial robots caused by sensor failures may interrupt the entire manufacturing process, thereby increasing production costs. In this paper, we first propose a domain-specific framework consisting of offline training and online inference to effectively detect anomalies in the scenario of industrial robotic sensors. In a nutshell, the framework can be identified in three folds: (1) the offline training obtaining historical data from the database to train the time series and anomaly detection models; (2) the online inference deploying the offline trained models for online anomaly detection in real-time; (3) the incremental learning updating the online model for a new type of anomalies. We then propose an improved Generative Adversarial Networks (GANs) named MSGAN with the adaptive update strategy mechanism based on WGAN-GP to generate fake anomaly samples, improving anomaly detection accuracy. Specifically, the Wasserstein distance with the gradient penalty is introduced to improve training stability and sample quality. Moreover, adapting the generator’s complexity in robotic sensors, an adaptive update strategy based on the loss change ratio is adopted to speed up training convergence. Extensive experiments based on real robotic sensor datasets demonstrate the effectiveness of the proposed framework. Moreover, the results demonstrate that the detection accuracy can be improved by the synthetic samples based on the proposed MSGAN algorithm.

With the proliferation of intelligent applications, mobile devices are increasingly handling computation-intensive tasks but often struggle with limited computing power and energy resources. Mobile Edge Computing (MEC) offers a solution by enabling these devices to offload computation-intensive tasks to resource-rich edge servers, thus reducing processing latency and energy consumption. However, existing task-offloading strategies often neglect critical security concerns. In this paper, we propose a security-aware task-offloading framework that utilizes Deep Reinforcement Learning (DRL) to solve these challenges. Our framework is designed to minimize the latency of task accomplishment and energy consumption while ensuring data security. We model system utility as a Markov Decision Process (MDP) and design a Proximal Policy Optimization (PPO)-based algorithm to derive optimal offloading strategies. Experimental results demonstrate that the proposed algorithm outperforms traditional methods regarding task execution latency and energy consumption.

In mobile edge networks (MENs), big data caching services are expected to provide mobile users with better quality of experience (QoE) than normal scenarios. However, the increasing types of sensors and devices are producing an explosion of big data. Extracting valuable contents for caching is becoming a vital issue for the satisfaction of QoE. Therefore, it is urgent to propose some rational strategies to improve QoE, which is the major challenge for content-centric caching. This article introduces a novel big data architecture consisting of data management units for content extraction and caching decision, improving quality of service and ensuring QoE. Then a caching strategy is proposed to improve QoE, including three parts: (1) the caching location decision, which means the method of deploying caching nodes to make them closer to users; (2) caching capacity assessment, which aims to seek suitable contents to match the capacity of caching nodes; and (3) caching priority choice, which leads to contents being cached according to their priority to meet user demands. With this architecture and strategy, we particularly use a caching algorithm based on deep reinforcement learning to achieve lower cost for intelligent caching. Experimental results indicate that our schemes achieve higher QoE than existing algorithms.

The Energy Internet is a fundamental infrastructure for deploying green city applications, where energy saving and job acceleration are two critical issues to address. In contrast to existing approaches that focus on static metrics with the assumption of complete prior knowledge of resource information, both application-level properties and energy-level requirements are realized in this paper by jointly considering energy saving and job acceleration during job runtime. Considering the online environment of smart city applications, the main objective is transferred as an optimization problem with a model partition and function assignment. To minimize the energy cost and job completion time together, a green workload placement approach is proposed by using the multi-action deep reinforcement learning method. Evaluations with real-world applications demonstrate the superiority of this method over state-of-the-art methods.

The multi-interface network is a promising underlying infrastructure to deploy distributed Deep Learning (DL) tasks as the gradients generated by a worker can be exchanged to others via different links in parallel. Although this decentralized parameter synchronization mechanism can reduce the time of gradient exchange, building a high-performance distributed DL architecture still requires the balance of communication efficiency and computation utilization, i.e., addressing the issues of traffic burst, data consistency and programming convenience. To achieve this goal, our idea is to asynchronously exchange gradient pieces without the central control in multi-interface network. We propose the Piece-level Gradient Exchange and Multi-interface Collective Communication to handle parameter synchronization and traffic transmission, respectively. Specifically, we design the gradient sketch approach based on 8-bit quantization to compress gradients and introduce colayer abstraction to better handle gradient partition, exchange and pipelining. Moreover, we provide uniform programming interfaces to capture the synchronization semantics and build the Gradient Exchange Index (GEI) data structures to make our approach online applicable. We implement our algorithms into a distributed DL system called Canary in PyTorch-1.4.0 and will open source of it. Experiments conducted in Alibaba Cloud demonstrate that Canary reduces 56:28% traffic on average and completes the training by up to 1:61×, 2:28× and 2:84× faster than BML, Ako on PyTorch, and PS on TensorFlow, respectively. Furthermore, evaluation results show that the gradient sketch method inside Canary can apply to state-of-the-art frameworks, e.g., Horovod and BytePS, to effectively reduce communication overhead and achieve higher training speed.