23 March 2020 in IEEE Transactions on Biomedical Circuits and Systems
The recovery of sparse signals given their linear mapping on lower-dimensional spaces can be partitioned into a support estimation phase and a coefficient estimation phase. We propose to estimate the support with an oracle based on a deep neural network trained jointly with the linear mapping at the encoder. The divination of the oracle is then used to estimate the coefficients by pseudo-inversion. This architecture allows the definition of an encoding-decoding scheme with state-of-the-art recovery capabilities when applied to biological signals such as ECG and EEG, thus allowing extremely low-complex encoders. As an additional feature, oracle-based recovery is able to self-assess, by indicating with remarkable accuracy chunks of signals that may have been reconstructed with a non-satisfactory quality. This self-assessment capability is unique in the CS literature and paves the way for further improvements depending on the requirements of the specific application. As an example, our scheme is able to satisfyingly compress by a factor of 2.67 an ECG or EEG signal with a complexity equivalent to only 24 signed sums per processed sample.
20 November 2020 in IEEE Journal on Emerging and Selected Topics in Circuits and Systems
We investigate the advantage of a two-step approach in the recovery of Compressed Sensing (CS) encoded signals in a realistic environment. First, the support of the signal is computed from the compressed measurements exploiting a Deep Neural Network (DNN). Once the support is known, the input signal can be easily recovered by a pseudoinverse operation. We consider a case study involving realistic biomedical signals and a processing architecture based on a limited precision fixed-point arithmetic unit for the implementation of the DNN and the pseudoinverse operation. In this setting, we show that the proposed approach results in a performance improvement of more than 5 dB in terms of average reconstructed signal to noise ratio (ARSNR) compared to CS state-of-the-art approach. This has been possible thanks to two main contributions reported in this paper. The first one is a theoretical investigation of the relationship between the definition of support and both the properties of the input signal and the adopted compression technique. The second one relies on replacing the pseudoinverse operation with a least mean square filter, whose small sensitivity to numerical errors grants advantages in architectures relying on limited precision fixed-point arithmetic units.
17 January 2025 in IEEE Transactions on Neural Networks and Learning Systems
The growing interest in the Internet of Things (IoT) and mobile artificial intelligence applications is pushing the investigation on deep neural networks (DNNs) that can operate at the edge using low-resources/energy devices. To obtain such a goal, several pruning techniques have been proposed in the literature. They aim to reduce the number of interconnections—and consequently the size, and the corresponding computing and storage requirements—of DNNs that traditionally rely on classic multiply-and-accumulate (MAC) neurons. In this work, we propose a novel neuron structure based on a multiply-and-max/min (MAM) map-reduce paradigm, and we show that by exploiting this new paradigm it is possible to build naturally and aggressively prunable DNN layers, with a negligible loss in performance. This novel structure allows a greater interconnection sparsity when compared to classic MAC-based DNN layers. Moreover, most of the already existing state-of-the-art pruning techniques can be used with MAM layers with little to no changes. To test the pruning performance of MAM, we employ different models—AlexNet, VGG-16 and the more recent ViT-B/16—and different computer vision datasets—CIFAR-10, CIFAR-100, and ImageNet-1K. Multiple pruning approaches are applied, ranging from single-shot methods to training-dependent and iterative techniques. As a notable example, we test MAM on the ViT-B/16 model fine-tuned on the ImageNet-1K task and apply one-shot gradient-based pruning. We remove interconnections until the model experiences a 6% decrease in accuracy. While the selected MAC-based layers need at least 38.2% remaining interconnections, MAM-based layers achieve the same accuracy with only 0.1%.
1 November 2020 in Integration