The non-negative matrix factorization (NMF) model with an additional orthogonality constraint on one of the factor matrices, called the orthogonal NMF (ONMF), has been found a promising clustering model and can outperform the classical K-means. However, solving the ONMF model is a challenging optimization problem because the coupling of the orthogonality and non-negativity constraints introduces a mixed combinatorial aspect into the problem due to the determination of the correct status of the variables (positive or zero). Most of the existing methods directly deal with the orthogonality constraint in its original form via various optimization techniques, but are not scalable for large-scale problems. In this paper, we propose a new ONMF based clustering formulation that equivalently transforms the orthogonality constraint into a set of norm-based non-convex equality constraints. We then apply a non-convex penalty (NCP) approach to add them to the objective as penalty terms, leading to a problem that is efficiently solvable. One smooth penalty formulation and one non-smooth penalty formulation are respectively studied. We build theoretical conditions for the penalized problems to provide feasible stationary solutions to the ONMF based clustering problem, as well as proposing efficient algorithms for solving the penalized problems of the two NCP methods. Experimental results based on both synthetic and real datasets are presented to show that the proposed NCP methods are computationally time efficient, and either match or outperform the existing K-means and ONMF based methods in terms of the clustering performance.

Federated learning (FL) has the potential to empower Internet-of-Vehicles (IoV) networks by enabling smart vehicles (SVs) to participate in the learning process under the orchestration of a vehicular service provider while keeping data locally. In this paper, we propose a novel federated stochastic primal-dual algorithm with differential privacy (FedSPD-DP) to ensure robust privacy protection for FL based IoV (FL-IoV) systems. The FedSPD-DP algorithm leverages multiple steps of local stochastic gradient descent (SGD) and partial client participation (PCP) to improve communication efficiency while incorporating differential privacy (DP) to ensure privacy protection. Our theoretical analysis explores the impact of these strategies on learning performance. Specifically, we demonstrate that the data sampling strategy and PCP enhance data privacy, while a larger number of local SGD steps may increase privacy leakage, revealing a non-trivial tradeoff between communication efficiency and privacy protection. Extensive experiments on real-world data validate the effectiveness of the proposed algorithm, showing superior performance compared to state-of-the-art FL algorithms, and confirming the analytical results and properties.

Recent demands on data privacy have called for federated learning (FL) as a new distributed learning paradigm in massive and heterogeneous networks. Although many FL algorithms have been proposed, few of them have considered the matrix factorization (MF) model, which is known to have a vast number of signal processing and machine learning applications. Since the MF problem involves two blocks of variables and the variables are usually subject to constraints related to specific solution structure, it requires new FL algorithm designs to achieve communication-efficient MF in heterogeneous data networks. In this paper, we address the challenge by proposing two new federated MF (FedMF) algorithms, namely, FedMAvg and FedMGS, based on the model averaging and gradient sharing principles, respectively. Both FedMAvg and FedMGS adopt multiple steps of local updates per communication round to speed up convergence, and allow only a randomly sampled subset of clients to communicate with the server for reducing the communication cost. Convergence properties for the two algorithms are thoroughly analyzed, which delineate the impacts of heterogeneous data distribution, local update number, and partial client communication on the algorithm performance, and guide the design of proposed algorithms. By focusing on a data clustering task, extensive experiment results are presented to examine the practical performance of proposed algorithms, as well as demonstrating their efficacy over the existing distributed clustering algorithms.

The burgeoning complexity of communication necessitates a high demand for security. Access control encryption is a promising primitive to meet the security demand but the bulk of its constructions rely on formulating the access control policy with identities. Attribute-based access control policy in attribute-based encryption (ABE) is known to be more expressive without relying on enumerating identities. We propose a generic framework to build attribute-based access control encryption from ciphertext-policy ABE. Our instantiations prioritize different emphases on expressiveness and efficiency. The first instantiation supports multi-valued AND-gate access control structures, while the second supports the linear-secret-sharing access structure. Both are prototyped with efficiency validated empirically.

Federated learning (FL) has been recognized as a viable solution for local-privacy-aware collaborative model training in wireless edge networks, but its practical deployment is hindered by the high communication overhead caused by frequent and costly server-device synchronization. Notably, most existing communication-efficient FL algorithms fail to reduce the significant inter-device variance resulting from the prevalent issue of device heterogeneity. This variance severely decelerates algorithm convergence, increasing communication overhead and making it more challenging to achieve a well-performed model. In this paper, we propose a novel communication-efficient FL algorithm, named FedQVR, which relies on a sophisticated variance-reduced scheme to achieve heterogeneity-robustness in the presence of quantized transmission and heterogeneous local updates among active edge devices. Comprehensive theoretical analysis justifies that FedQVR is inherently resilient to device heterogeneity and has a comparable convergence rate even with a small number of quantization bits, yielding significant communication savings. Besides, considering non-ideal wireless channels, we propose FedQVR-E which enhances the convergence of FedQVR by performing joint allocation of bandwidth and quantization bits across devices under constrained transmission delays. Extensive experimental results are also presented to demonstrate the superior performance of the proposed algorithms over their counterparts in terms of both communication efficiency and application performance.