On-chip implementation of large-scale neural networks with emerging synaptic devices is attractive but challenging, primarily due to the pre-mature analog properties of today's resistive memory technologies. This work aims to realize a large-scale neural network using today's available binary RRAM devices for image recognition. We propose a methodology to binarize the neural network parameters with a goal of reducing the precision of weights and neurons to 1-bit for classification and <;8-bit for online training. We experimentally demonstrate the binary neural network (BNN) on Tsinghua's 16 Mb RRAM macro chip fabricated in 130 nm CMOS process. Even under finite bit yield and endurance cycles, the system performance on MNIST handwritten digit dataset achieves ~96.5% accuracy for both classification and online training, close to ~97% accuracy by the ideal software implementation. This work reports the largest scale of the synaptic arrays and achieved the highest accuracy so far.

Recently, 2-D cross-point array of resistive random access memory (RRAM) has been proposed for implementing the weighted sum and weight update operations to accelerate the neuro-inspired learning algorithms on chip. This paper aims to extend such 2-D cross-point array to 3-D vertical array for storing and computing the large-scale weight matrices in the neural network. Considering the fabrication and 3-D integration of analog synapses (i.e., multilevel RRAM devices) are premature at this stage, we propose using today's available digital or binary RRAM devices for implementing a ternary neural network, which aggressively reduces the weight precision to ternary levels (+1, 0,-1) for the weighted sum in both feedforward and backward inference, while the multiple 3-D layers could serve for accumulating the small errors in a higher precision format for weight update. Compared to the 2-D implementation, the proposed 3-D vertical implementation shows larger read/write margin for weighted sum/weight update, smaller latency, and energy consumption for weight update. This paper demonstrates the attractiveness for building a monolithic 3-D neuromorphic hardware platform.

The 3-D integration of resistive switching random access memory (RRAM) array is attractive for low-cost and high-density nonvolatile memory application. In this paper, the design tradeoffs of select transistor drivability, RRAM device characteristics, such as switching current (I _W ), ON/OFF-state resistance (R _ON /R _OFF ), and I-V nonlinearity ratio, interconnect material, and write/read scheme are systematically analyzed using a 3-D circuit simulation. The simulation results show that insufficient current drivability of the vertical transistor severely limits the number of 3-D layers. A low switching current (high R _ON ) or a high nonlinearity is beneficial for improving the write margin, while it degrades the read current sense margin. To alleviate this conflict, the read voltage needs to be boosted to the half write voltage. The common RRAM electrode material TiN is not suitable for the interconnect material due to a high resistivity. To improve write energy efficiency, a multiple-bit write scheme is proposed to reduce the write energy consumption per bit and enable a high bandwidth. With R _ON = 500 kΩ (I _W = 6 μA) and nonlinearity ratio = 10, 1-Mb 3-D vertical RRAM subarray is feasible to meet the specified write/read margin with ~2-pJ/bit energy consumption.

Artificial neural networks (ANNs) have led to landmark changes in many fields, but they still differ significantly from the mechanisms of real biological neural networks and face problems such as high computing costs, excessive computing power, and so on. Spiking neural networks (SNNs) provide a new approach combined with brain-like science to improve the computational energy efficiency, computational architecture, and biological credibility of current deep learning applications. In the early stage of development, its poor performance hindered the application of SNNs in real-world scenarios. In recent years, SNNs have made great progress in computational performance and practicability compared with the earlier research results, and are continuously producing significant results. Although there are already many pieces of literature on SNNs, there is still a lack of comprehensive review on SNNs from the perspective of improving performance and practicality as well as incorporating the latest research results. Starting from this issue, this paper elaborates on SNNs along the complete usage process of SNNs including network construction, data processing, model training, development, and deployment, aiming to provide more comprehensive and practical guidance to promote the development of SNNs. Therefore, the connotation and development status of SNN computing is reviewed systematically and comprehensively from four aspects: composition structure, data set, learning algorithm, software/hardware development platform. Then the development characteristics of SNNs in intelligent computing are summarized, the current challenges of SNNs are discussed and the future development directions are also prospected. Our research shows that in the fields of machine learning and intelligent computing, SNNs have comparable network scale and performance to ANNs and the ability to challenge large datasets and a variety of tasks. The advantages of SNNs over ANNs in terms of energy efficiency and spatial-temporal data processing have been more fully exploited. And the development of programming and deployment tools has lowered the threshold for the use of SNNs. SNNs show a broad development prospect for brain-like computing.

The RRAM-based accelerators have become very popular candidates for neural network acceleration due to they perform matrix-vector multiplication in-memory with high storage density and low latency. Many related works have used fixed-precision quantization to achieve the model compression and enhance the tolerance of process variation, but these methods still suffer a large accuracy degradation and poor robustness to nonideal effects. In this work, we propose a crossbar-aware mixed-precision quantization scheme, which enables to search for the optimal precision of each part of the network as a way to improve the accuracy and robustness to noise. First, we introduce a group quantization strategy that can flexibly adjust the group size dynamically according to the crossbar size. Then, we propose a detailed mixed-precision search flow to search for the optimal precision set of the network. Finally, we give a noise injection adaption training method to enhance the tolerance of noise. Experimental results show that our proposed method can improve inference accuracy by at least 2.04% compared to the fixed-precision quantization under the same resource cost. The searched architecture with the highest accuracy most accurate (MA) could achieve an accuracy of 92.39% and a resource saving of 93.30% compared to the full precision model. The searched architecture with the biggest resource savings most efficient (ME) could achieve an accuracy of 91.11% and a resource saving of 95.57% compared to the full precision model. And, the average precision of the ME mixed-precision architecture is only 1.4 bits. Besides, the results show the mixed-precision network with noise adaption training is more robust to noise than the fixed-precision network with noise adaption training.

Epilepsy is a neurological disorder characterized by abnormal neuronal discharges in the brain. As a rich source of biometric information, electroencephalography (EEG) provides favorable conditions for automated detection. Traditional algorithms and manual analysis possess solid theoretical foundations and good interpretability, however, these methods predominantly require extensive domain expertise and involve lengthy processing pipelines for complex data. The advent of artificial intelligence (AI) has facilitated the application of neural networks in the detection and prediction of epilepsy. Although such approaches heavily rely on high-quality annotated data, suffer from limited model interpretability, and involve complex training and parameter tuning, these efficient, real-time, end-to-end models still demonstrate significant potential in epilepsy analysis. This review systematically analyzes and summarizes the neural network technologies used in 341 papers published in the past three years, employing the PRISMA standard procedure. To facilitate readers’ related research, the review also summarizes the basic information of 16 publicly available datasets, common features, and metrics. Specifically, this review offers a comprehensive evaluation of diverse neural network architectures, concluding that convolutional neural networks have become a prevalent choice as classic neural networks. Furthermore, graph neural networks and transformers are experiencing a marked surge in popularity. The application of hybrid neural networks to fully extract information from EEG is also a growing trend. The review concludes with a comprehensive discussion and summary of the technical characteristics, research directions, and limitations of current methods, including patient-to-patient identification, explainable AI, dataset bias, and zone location.