High-dimensional crowdsourced data collected from numerous users produces rich knowledge about our society; however, it also brings unprecedented privacy threats to the participants. Local differential privacy (LDP), a variant of differential privacy, is recently proposed as a state-of-the-art privacy notion. Unfortunately, achieving LDP on high-dimensional crowdsourced data publication raises great challenges in terms of both computational efficiency and data utility. To this end, based on the expectation maximization (EM) algorithm and Lasso regression, we first propose efficient multi-dimensional joint distribution estimation algorithms with LDP. Then, we develop a local differentially private high-dimensional data publication algorithm (LoPub) by taking advantage of our distribution estimation techniques. In particular, correlations among multiple attributes are identified to reduce the dimensionality of crowdsourced data, thus speeding up the distribution learning process and achieving high data utility. Extensive experiments on real-world datasets demonstrate that our multivariate distribution estimation scheme significantly outperforms existing estimation schemes in terms of both communication overhead and estimation speed. Moreover, LoPub can keep, on average, 80% and 60% accuracy over the released datasets in terms of support vector machine and random forest classification, respectively.

Federated learning (FL) is an emerging privacy-preserving paradigm that enables multiple participants collaboratively to train a global model without uploading raw data. Considering heterogeneous computing and communication capabilities of different participants, asynchronous FL can avoid the stragglers effect in synchronous FL and adapts to scenarios with vast participants. Both staleness and non-IID data in asynchronous FL would reduce the model utility. However, there exists an inherent contradiction between the solutions to the two problems. That is, mitigating the staleness requires to select less but consistent gradients while coping with non-IID data demands more comprehensive gradients. To address the dilemma, this paper proposes a two-stage weighted $K$ asynchronous FL with adaptive learning rate (WKAFL). By selecting consistent gradients and adjusting learning rate adaptively, WKAFL utilizes stale gradients and mitigates the impact of non-IID data, which can achieve multifaceted enhancement in training speed, prediction accuracy and training stability. We also present the convergence analysis for WKAFL under the assumption of unbounded staleness to understand the impact of staleness and non-IID data. Experiments implemented on both benchmark and synthetic FL datasets show that WKAFL has better overall performance compared to existing algorithms.

The massive generation, extensive sharing, and deep exploitation of data in the big data era have raised unprecedented privacy threats. To address privacy concerns, various privacy paradigms have been proposed to achieve a good tradeoff between privacy and data utility. Particularly, differential privacy has been well accepted as one of the de facto standard for privacy preservation, and numerous schemes guaranteeing differential privacy have been proposed. Nonetheless, most of the existing works claiming a superior utility-privacy tradeoff only present specific methods, with distinct perspectives, and a complete comparative analysis and evaluation study has not been fully investigated. To this end, in this paper we review and investigate existing schemes on providing differential privacy from a broad and encompassing perspective to provide a comprehensive survey with respect to both the privacy guarantee and the effectiveness and efficiency in utility improvement. We categorize the existing schemes into distribution optimization, sensitivity calibration, transformation, decomposition, and correlations exploitation, based on their mechanisms in improving data utility. We also conduct some analysis and comparison of their various concepts and principles, focusing on improvements to data utility. Finally, we outline some challenges and provide future research directions.

Latency-aware service placement is promising in reducing the overall service response latency of proliferating edge-cloud collaborative smart manufacturing systems. However, intuitive latency estimators used by existing service placement approaches cannot accurately depict the nonlinear end-to-end (E2E) latency of multihop microservices with complex dependencies, which is severely hindering the effectiveness of latency-aware service placement. To address this issue, in this article, we present a microservice placement mechanism for edge-cloud collaborative smart manufacturing (MPCSM), where a microservice placement algorithm latency-aware edge-cloud collaborative placement supported by an accurate data-driven E2E latency estimation method is proposed. We build a real-world collaborative prototype, and conduct a case study on semiconductor manufacturing to elaborate the construction of our latency estimator. Results of extensive experiments demonstrate that the error of our E2E latency estimator is up to 10× less than that of existing ones, and the overall service latency with MPCSM is up to 10× less than that with existing service placement approaches.

An edge-cloud collaborative intelligent (ECCI) platform is of great significance for the agile development and rapid deployment of ECCI applications, which are essential for realizing smart industry in the era of Industry 4.0. However, the existing platforms lack considering the high real-time latency demand of industrial operations, which severely hinders the development of smart industry and may even lead to severe industrial accidents. To effectively reduce the response latency of industrial applications, in this article, we propose an ECCI platform IndustEdge. It takes time-sensitive networking as the deterministic transport for the link layer, and provides an extensible ECCI orchestration component to reduce the system level latency. Furthermore, IndustEdge has an ECCI algorithm library for different collaborative modes and provides the complete life cycle management for ECCI applications. We implement platforms for both the real-world prototype and emulated-world emulation, and conduct two case studies to evaluate the effectiveness of IndustEdge.

The fusion of federated learning and differential privacy can provide more comprehensive and rigorous privacy protection, thus attracting extensive interests from both academia and industry. However, facing the system-level challenge of device heterogeneity, most current synchronous FL paradigms exhibit low efficiency due to the straggler effect, which can be significantly reduced by Asynchronous FL (AFL). However, AFL has never been comprehensively studied, which imposes a major challenge in the utility optimization of DP-enhanced AFL. Here, theoretically motivated multi-stage adaptive private algorithms are proposed to improve the trade-off between model utility and privacy for DP-enhanced AFL. In particular, we first build two DP-enhanced AFL frameworks with consideration of universal factors for different adversary models. Then, we give a solid analysis on the model convergence of AFL, based on which, DP can be adaptively achieved with high utility. Through extensive experiments on different training models and benchmark datasets, we demonstrate that the proposed algorithms achieve the overall best performances and improve up to 24% test accuracy with the same privacy loss and have faster convergence compared with the state-of-the-art algorithms. Our frameworks provide an analytical way for private AFL and adapt to more complex FL application scenarios.