16 October 2024 in IEEE Transactions on Circuits and Systems for Artificial Intelligence
Machine learning algorithms, especially deep neural networks (DNNs), are computationally expensive and vulnerable to privacy breaches. Hyperdimensional Computing (HDC) is a brain-inspired machine learning paradigm which offers a potential alternative for efficient and private learning. HDC encodes dense, sensory inputs into sparse, high-dimensional vectors of 10,000 bits or more, and learns using well-defined and lightweight operations on these vectors. However, naïve HDC is vulnerable to privacy attacks as the encoding process can be reversely determined. In this paper, we systematically study the fundamental privacy challenges of HDC due to reversibility and propose an end-to-end private training and inference framework for HDC with efficient hardware implementation. We first show that HDC is not naturally private as the original data and encoding parameters can be reconstructed from the encoded vector, regardless of the encoding methods applied. We term this attack as the feature extraction attack. We then propose a locally sparse encoding method to obfuscate the encoded information for protecting inference privacy, leading to computational and communication savings. We further propose Privé -HDnn which uses a hybrid architecture of Convolutional Neural Network (CNN) as feature extractors and private-preserved HDC classifier for complex tasks such as image classification. Thanks to the noise tolerance of HDC, Privé-HDnn enables private single- and multi-pass training of HDC by differential privacy-guaranteed noise injection. Results show that our new encoding method defends against decoding attacks with an RMSE deviation of 0.42, while also speeding up encoding by 2.6-5.3× (19-79×) compared to the FPGA (CPU) baseline. For privacy-preserved training, Privé - HDnn achieves comparable or up to 5.8% better accuracy than DNNs with up to 1231× faster training time on GPU as compared to the state-of-the-art NNs.
10 July 2024 in IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
Processing in-memory (PIM) is promising to accelerate neural networks (NNs) because it minimizes data movement and provides large computational parallelism. Similar to machine learning accelerators, application mapping, which determines the operation scheduling and data layout, plays a critical role in the NN acceleration on PIM. The mapping optimization of previous NN accelerators focused on optimizing the latency of sequential execution. However, PIM accelerators feature a distinct design space of application mapping from conventional NN accelerators, due to the spatial execution of NN layers across different memory locations. This enables opportunities for overlapping execution of consecutive NN layers to improve the latency, where the succeeding layer can start execution before the preceding layer fully completes the computation. In this paper, we propose framework that incorporates the computational overlapping optimization into the DNN mapping exploration process on PIM architectures. includes analytical algorithms for fast and accurate overlap analysis. Furthermore, it proposes a novel mapping search strategy and a transformation mechanism to enable efficient design space exploration on the overlap-based mapping for the whole network. Our framework demonstrates a significant improvement in runtime performance from 3.4× to 323.1× compared to the previous state-of-the-art overlap-based framework. Our experiments show that Fast-OverlaPIM can efficiently produce mappings that are 4.6× to 18.1× faster than the state-of-the-art mapping optimization framework under the same architecture constraints.