The performance of a biologically plausible spiking neural network (SNN) largely depends on the model parameters and neural dynamics. This article proposes a parameter optimization scheme for improving the performance of a biologically plausible SNN and a parallel on-field-programmable gate array (FPGA) online learning neuromorphic platform for the digital implementation based on two numerical methods, namely, the Euler and third-order Runge–Kutta (RK3) methods. The optimization scheme explores the impact of biological time constants on information transmission in the SNN and improves the convergence rate of the SNN on digit recognition with a suitable choice of the time constants. The parallel digital implementation leads to a significant speedup over software simulation on a general-purpose CPU. The parallel implementation with the Euler method enables around $180\times $ ( $20\times $ ) training (inference) speedup over a Pytorch-based SNN simulation on CPU. Moreover, compared with previous work, our parallel implementation shows more than $300\times $ ( $240\times $ ) improvement on speed and $180\times $ ( $250\times $ ) reduction in energy consumption for training (inference). In addition, due to the high-order accuracy, the RK3 method is demonstrated to gain $2\times $ training speedup over the Euler method, which makes it suitable for online training in real-time applications.

The bottleneck between the processing elements and memory is the biggest issue contributing to the scalability problem in computing. In-memory computation is an alternative approach that combines memory and processor in the same location, and eliminates the potential memory bottlenecks. Associative processors are a promising candidate for in-memory computation, however the existing implementations have been deemed too costly and power hungry. Approximate computing is another promising approach for energy-efficient digital system designs where it sacrifices the accuracy for the sake of energy reduction and speedup in error-resilient applications. In this study, approximate in-memory computing is introduced in memristive associative processors. Two approximate computing methodologies are proposed; bit trimming and memristance scaling. Results show that the proposed methods not only reduce energy consumption of in-memory parallel computing but also improve their performance. As compared to other existing approximate computing methodologies on different architectures (e.g., CPU, GPU, and ASIC), approximate memristive in-memory computing exhibits better results in terms of energy reduction (up to 80x) and speedup (up to 20x) on a variety of benchmarks from different domains when quality degradation is limited to 10% and it confirms that memristive associative processors provide a highly-promising platform for approximate computing.

In-memory computing is an emerging computing paradigm that overcomes the limitations of exiting Von-Neumann computing architectures such as the memory-wall bottleneck. In such paradigm, the computations are performed directly on the data stored in the memory, which highly reduces the memory-processor communications during computation. Hence, significant speedup and energy savings could be achieved especially with data-intensive applications. Associative processors (APs) were proposed in the seventies and recently were revived thanks to the high-density memories. In this tutorial brief, we overview the functionalities and recent trends of APs in addition to the implementation of each content-addressable memory with different technologies. The AP operations and runtime complexity are also summarized. We also explain and explore the possible applications that can benefit from APs. Finally, the AP limitations, challenges, and future directions are discussed.

The complexity of the computational problems is rising faster than the computational platforms’ capabilities which are also becoming increasingly costly to operation due to their increased need for energy. This forces researchers to find alternative paradigms and methods for efficient computing. One promising paradigm is accelerating compute intensive kernels using in-memory computing accelerators where data movements are significantly reduced. Another increasingly popular method for improving energy efficiency is approximate computing. In this study, we propose a methodology for efficient approximate in-memory computing. To maximize energy savings for a given approximation constraints, a hybrid approach is presented combining both voltage and precision scaling. This can be applied to an associative memory based architecture that can be implemented today using CMOS memories (SRAM) but can be seamlessly scaled to emerging ReRAM-based memory technology later with minimal effort. For the evaluation of the proposed methodology, a diverse set of domains is covered such as image processing, machine learning, machine vision, digital signal processing, etc. When compared to full-precision, unscaled implementations, average energy savings of 5.17x and 59.11x, and speedups of 2.1x and 3.24x in SRAM-based and ReRAM-based architectures, respectively are reported.

To solve real-time challenges, neuromorphic systems generally require deep and complex network structures. Thus, it is crucial to search for effective solutions that can reduce network complexity, improve energy efficiency, and maintain high accuracy. To this end, we propose unsupervised pruning strategies that are focused on pruning neurons while training in spiking neural networks (SNNs) by utilizing network dynamics. The importance of neurons is determined by the fact that neurons that fire more spikes contribute more to network performance. Based on these criteria, we demonstrate that pruning with an adaptive spike count threshold provides a simple and effective approach that can reduce network size significantly and maintain high classification accuracy. The online adaptive pruning shows potential for developing energy-efficient training techniques due to less memory access and less weight-update computation. Furthermore, a parallel digital implementation scheme is proposed to implement spiking neural networks (SNNs) on field programmable gate array (FPGA). Notably, our proposed pruning strategies preserve the dense format of weight matrices, so the implementation architecture remains the same after network compression. The adaptive pruning strategy enables 2.3× reduction in memory size and 2.8× improvement on energy efficiency when 400 neurons are pruned from an 800-neuron network, while the loss of classification accuracy is 1.69%. And the best choice of pruning percentage depends on the trade-off among accuracy, memory, and energy. Therefore, this work offers a promising solution for effective network compression and energy-efficient hardware implementation of neuromorphic systems in real-time applications.

Computing based on traditional approaches has hit its limits in terms of scalability and power consumption. For this reason, new computing paradigms have started to emerge that aim to perform operations directly in-memory to eliminate data movement costs. Even though traditional in-memory processor architectures together with emerging semiconductor technologies show promise for improving the efficiency of parallel computing, they lack some vital requirements such as flexibility and sequential execution. In this paper, a novel 2-D in-memory computing architecture is proposed. The proposed associative processing architecture is implemented by both CMOS/static random-access memory and ReRAM technologies and employed as an accelerator. As proven by circuit simulations and comparisons on a variety of benchmarks from different domains, the proposed architecture facilitates very efficient in-memory parallel computing together with a high degree of flexibility that results in faster running time in fundamental benchmarks.