Reinforcement learning is a learning paradigm for solving sequential decision-making problems. Recent years have witnessed remarkable progress in reinforcement learning upon the fast development of deep neural networks. Along with the promising prospects of reinforcement learning in numerous domains such as robotics and game-playing, transfer learning has arisen to tackle various challenges faced by reinforcement learning, by transferring knowledge from external expertise to facilitate the efficiency and effectiveness of the learning process. In this survey, we systematically investigate the recent progress of transfer learning approaches in the context of deep reinforcement learning. Specifically, we provide a framework for categorizing the state-of-the-art transfer learning approaches, under which we analyze their goals, methodologies, compatible reinforcement learning backbones, and practical applications. We also draw connections between transfer learning and other relevant topics from the reinforcement learning perspective and explore their potential challenges that await future research progress.

The ultra-low latency requirements of 5G/6G applications and privacy constraints call for distributed machine learning systems to be deployed at the edge. With its simple yet effective approach, federated learning (FL) is a natural solution for massive user-owned devices in edge computing with distributed and private training data. FL methods based on FedAvg typically follow a naive star topology, ignoring the heterogeneity and hierarchy of the volatile edge computing architectures and topologies in reality. Several other network topologies exist and can address the limitations and bottlenecks of the star topology. This motivates us to survey network topology-related FL solutions. In this paper, we conduct a comprehensive survey of the existing FL works focusing on network topologies. After a brief overview of FL and edge computing networks, we discuss various edge network topologies as well as their advantages and disadvantages. Lastly, we discuss the remaining challenges and future works for applying FL to topology-specific edge networks.

Recently, handwriting recognition approaches has been widely applied to HCI (Human-Computer Interface) applications. The emergence of the novel mobile terminals urges a more man-machine friendly interface mode. The previous airwriting recognition approaches have been accomplished by virtue of cameras and sensors. However, the vision based approaches are susceptible to the light condition and sensor based methods have disadvantages in deployment and highcost. The latest researches have demonstrated that the pervasive wireless signals can be used to identify different gestures. In this paper, we attempt to utilize CSI (channel state information) derived from wireless signals to realize the device-free air-write recognition called Wri-Fi. Compared to the gesture recognition,the increased diversity and complexity of characters of the alphabet make it challenging. The PCA (Principle Component Analysis) is used for denoising effectively and the energy indicator derived from the FFT (Fast Fourier Transform) is to detect action continuously. The unique CSI waveform caused by unique writing patterns of 26 letters serve as feature space. Finally, the HMM (Hidden Markov model) is used for character modeling and classification. We conduct experiments in our laboratory and get the average accuracy of the Wri-Fi are 86.75% and 88.74% in two writing areas, respectively.

Cloud providers have started to deploy various FPGA accelerators in their datacenters because the performance of many applications can be significantly improved by implementing their core routines in FPGAs. In conventional datacenters with FPGA accelerated servers, if a tenant wants to use FPGA accelerators, it requests for a VM instance residing in a server equipped with an FPGA accelerator. This paradigm to integrate FPGA into Cloud leads to poor resource sharing of the precious FPGA resources. In this paper, we propose FPGAPooling, an FPAG-enabled Cloud system where all FPGA accelerators are managed as a single resource pool and shared among all VMs. For a VM, instead of requesting the Cloud to run the VM on an FPGA accelerated server, at runtime, when a VM needs to use FPGA acceleration, it requests an FPGA accelerator from the pool. We design a centralized scheduler to handle acceleration requests at runtime; We implemented a system prototype on IBM's OpenPower Cloud system. We designed and implemented a group of scheduling algorithms for the FPGAPooling system. With extensive evaluations, we found that our algorithms can improve the average and tail job completion time by up to 7 and 4 times, respectively.

Mobile devices have emerged as the most popular platforms to access information. However, they have also become a major concern of privacy violation. In this paper, we propose MagAttack, which exploits the electromagnetic side channel of a laptop to guess user activities, i.e., application launching and application operation. The key insight of MagAttack is that applications are discrepant in essence due to the different compositions of instructions, which can be reected on the CPU power consumption, and thus the corresponding EM emissions. MagAttack is challenging since that EM signals are noisy due to the dynamics of applications and the limited sampling rate of the built-in magnetometers in COTS mobile devices. We overcome these challenges and convert noisy coarse-grained EM signals to robust ne-grained features. We implement MagAttack on both an iOS and an Android smartphone without any hardware modication, and evaluate its performance with 13 popular applications, 15 YouTube videos, and 50 top websites in China. The results demonstrate that MagAttack can recognize aforementioned 13 applications with an average accuracy of 99.5%, and gure out the playing operation among 15 videos with an average accuracy of 99.5% and the visiting operation among 50 websites with an average accuracy of 90.4%.