Federated learning, as one enabling technology of edge intelligence, has gained substantial attention due to its efficacy in training deep learning models without data privacy and network bandwidth concerns. However, due to the heterogeneity of the edge computing system and data, many methods suffer from the “ client-drift ” issue that could considerably impede the convergence of global model training: local models on clients can drift apart, and the aggregated model can be different from the global optimum. To tackle this issue, one intuitive idea is to guide the local model training by global teachers, i.e. , past global models, where each client learns the global knowledge from past global models via adaptive knowledge distillation techniques. Inspired by these insights, we propose a novel approach for heterogeneous federated learning, F ed GKD, which fuses the knowledge from historical global models and guides local training to alleviate the “ client-drift ” issue. In this paper, we evaluate F ed GKD through extensive experiments across various CV and NLP datasets (i.e., CIFAR-10/100, Tiny-ImageNet, AG News, SST5) under different heterogeneous settings. The proposed method is guaranteed to converge under common assumptions and outperforms the state-of-the-art baselines in the non-IID federated setting.

Internet-of-Things-enabled E-health system, which could monitor and collect the personal health information (PHI), has gradually transformed the clinical treatment to a more personalized way with in-home monitoring smart devices. Then, with the collected PHI, clinical decision support systems (CDSSs), which are based on data mining techniques and historical electronic medical records (EMRs) to help clinicians make proper treatment decisions, have attracted considerable attention. To address issues, such as network congestion and low rate of responsiveness for traditional methods when implementing CDSSs, we integrate the technologies mobile-edge computing (MEC) and software-defined networking for exploiting the computation resources and storage capacities among edge nodes (ENs) (i.e., MEC servers) in our model. Based on this integrated system, each edge node will deploy a double deep Q-network (DDQN) to obtain a stable and sequential clinical treatment policy. It is enabled by a novel fully decentralized federated framework (FDFF) for aggregating models of DDQN and extracting the knowledge from EMRs across all ENs. Furthermore, we discuss the convergence of FDFF in resource-constrained environments. However, since most EMRs are faced with stringent privacy concerns, we adopt two additively homomorphic encryption schemes to prevent leakage of EMRs' privacy during the training process of FDFF. Finally, we measure the time cost of our additively homomorphic encryption schemes and validate DDQN with experiments on large data sets based on FDFF, which shows promising performance on clinician treatment.

Cloud computing provides individuals and enterprises massive computing power and scalable storage capacities to support a variety of big data applications in domains like health care and scientific research, therefore more and more data owners are involved to outsource their data on cloud servers for great convenience in data management and mining. However, data sets like health records in electronic documents usually contain sensitive information, which brings about privacy concerns if the documents are released or shared to partially untrusted third-parties in cloud. A practical and widely used technique for data privacy preservation is to encrypt data before outsourcing to the cloud servers, which however reduces the data utility and makes many traditional data analytic operators like keyword-based top- $k$ document retrieval obsolete. In this paper, we investigate the multi-keyword top- $k$ search problem for big data encryption against privacy breaches, and attempt to identify an efficient and secure solution to this problem. Specifically, for the privacy concern of query data, we construct a special tree-based index structure and design a random traversal algorithm, which makes even the same query to produce different visiting paths on the index, and can also maintain the accuracy of queries unchanged under stronger privacy. For improving the query efficiency, we propose a group multi-keyword top- $k$ search scheme based on the idea of partition, where a group of tree-based indexes are constructed for all documents. Finally, we combine these methods together into an efficient and secure approach to address our proposed top- $k$ similarity search. Extensive experimental results on real-life data sets demonstrate that our proposed approach can significantly improve the capability of defending the privacy breaches, the scalability and the time efficiency of query processing over the state-of-the-art methods.

Federated learning (FL) has attracted increasing attention in recent years due to its data privacy preservation and great applicability to large-scale user scenarios. However, when FL faces numerous clients, it is inevitable to emerge the non-independent and identically distributed (non-iid) data between clients, which brings an enormous challenge for model training and performance analysis like convergence. Besides, due to the non-iid data, the participating clients of FL tend to be extremely heterogeneous so the number of samplings among clients causes a sampling variance problem, which induces a huge variation in convergence. More importantly, although FL can foster privacy security via locally retaining the training data, if local data is secret and sensitive, FL should have more powerful privacy protection to resist the cloud server or third party to infer private information from shared models or intermediate gradients. Facing the non-iid and privacy challenges, we propose a differential privacy (DP) based non-iid FL algorithm called DPNFL to jointly tackle these two issues. Specifically, motivated by the DP and its variants, we are the first to adopt the truncated concentrated differential privacy technique under the FL scenario to more tightly track end-to-end privacy loss, while requiring less noise injection for the same level of DP. To avoid the sampling variance problem, we enable the server to sample the partial clients uniformly without replacement, which also guarantees unbiased sampling. To further improve the algorithm performance, we also propose an adaptive version of DPNFL named AdDPNFL, which adopts the adaptive optimization on the server-side to simultaneously alleviate the impact of non-iid data and DP noise on model utility. Finally, we perform extensive experiments to validate the effectiveness and superiority of our algorithms.

One existing challenge associated with large scale skyline queries on cloud services, particularly when dealing with private information such as biomedical data, is supporting multi-party queries with curious-but-honest parties on encrypted data. In addition, existing solutions designed for performing secure skyline queries incur significant communication and computation costs due to ciphertext calculation. Thus, in this paper, we demonstrate the potential of supporting privacy-preserving multi-party skyline queries on encrypted data using additive homomorphic and proxy re-encryption cryptosystems. However, the secure computation based on these cryptosystems will further slow down query efficiency. To improve the efficiency of comparison on encrypted data, we redesign two lightweight secure comparison protocols. Meanwhile, we present an efficient method named “blind-reading” to securely obtain the skyline point. We also propose a novel method, Privacy Matrix, designed to reduce the scale of the dataset so that the computational cost is significantly decreased without privacy leakage. Then, we construct our secure skyline query protocol by integrating lightweight secure comparison protocols, “blind-reading” and Privacy Matrix techniques. Finally, we evaluate the security of our protocol, where we show it is secure without leaking information. The performance evaluation also shows that our proposed approach significantly improves the efficiency (at least ×4.5 faster) compared to the-state-of-art and has the scalability of query processing under large datasets.

Deep learning (DL) has been adopted in a broad range of Internet-of-Things (IoT) applications such as auto-driving, intelligent healthcare and smart grids, but limitations such as those relating to data and user privacy can complicate its broader implementation. Seeking to jointly address both privacy and utility, in this paper we connect the layer-wise relevance propagation with gradient descent for injecting proper noise into gradients. We also improve the conventional gradient clipping method by dividing the gradients into several groups; thus, minimizing the gradient distortion. Since the noisy gradient causes the undetermined descent direction and might adversely affect the loss minimization, we use the NoisyMin algorithm to select the best step size for each gradient perturbation. Finally, we integrate the adaptive optimizer into the gradient descent. In addition to improving the model utility, we also leverage the leading Sinh-Normal noise addition mechanism to achieve truncated concentrated differential privacy (tCDP) – as demonstrated by our rigorous analysis. Our experimental evaluations also validate the effectiveness of the proposed algorithm.