Multiply-accumulate calculations using a memristor crossbar array is an important method to realize neuromorphic computing. However, the memristor array fabrication technology is still immature, and it is difficult to fabricate large-scale arrays with high-yield, which restricts the development of memristor-based neuromorphic computing technology. Therefore, cascading small-scale arrays to achieve the neuromorphic computational ability that can be achieved by large-scale arrays, which is of great significance for promoting the application of memristor-based neuromorphic computing. To address this issue, we present a memristor-based cascaded framework with some basic computation units, several neural network processing units can be cascaded by this means to improve the processing capability of the dataset. Besides, we introduce a split method to reduce pressure of input terminal. Compared with VGGNet and GoogLeNet, the proposed cascaded framework can achieve 93.54% Fashion-MNIST accuracy and 86.64% CIFAR-10 accuracy under the 4.15M parameters. Extensive experiments with Ti/AlOx/TaOx/Pt we fabricated are conducted to show that the circuit simulation results can still provide a high recognition accuracy, and the recognition accuracy loss after circuit simulation can be controlled at around 0.39%.

As one of the most promising methods in the next generation of neuromorphic systems, memristor-based spiking neural networks (SNNs) show great advantages in terms of power efficiency, integration density, and biological plausibility. However, because of the nondifferentiability of discrete spikes, it is difficult to train SNNs with gradient descent and error backpropagation online. In this article, we propose an improved training algorithm for multilayer memristive SNN (MSNN) with three methods spontaneously, supporting in situ learning in hardware: 1) temporal order encoding is applied to generate different pulse trains in neurons; 2) a simplified homeostasis is realized by the activation state and refractory period to regulate hidden neurons spontaneously; and 3) spiking-timing-dependent plasticity (STDP) in memristive synapses is adopted to update weights in situ . Correspondingly, we provide a circuitry example and verify it in LTSPICE. Then the MSNN is benchmarked with the MNIST data set and analyzed with visualization methods, showing better recognition accuracy (95.15%) than existing SNNs with comparable scales and bio-inspired learning rules. We also consider some nonideal effects in memristor crossbar array and peripheral circuits. Evaluation results show that the proposed MSNN is robust to finite resolution, circuit noise and writing noise; and larger network scale will help the MSNN alleviate the negative impacts of other nonideal factors, including yield and device-to-device variation. Moreover, the energy efficiency of a MSNN system is estimated to achieve 7.6TOPS/W, showing great potential in low-power applications.

Many mainstream applications require multiply-accumulate calculations, such as image processing and neuromorphic computing. Multiply-accumulate calculations using memristor crossbar arrays is a remarkable method for extremely high implementation efficiency, whereas the memristor array fabrication technology is still not mature and it is difficult to fabricate large-scale arrays with high-yield, which will seriously affect the performance of the application running on the RRAM crossbar. This paper proposes an inputs split based calibration method that improves the application accuracy in tolerating variations and stuck-at-fault of memristor devices. To demonstrate the performance of the calibration algorithm, the case of image sharpening and three neural networks architectures are applied for simulation experiments. And experimental results show that the calculation accuracy can be improved by up to 26.83% at 90% yield of crossbar arrays, and the success rate of the algorithm can be as high as 99.3% when there are several arrays cascaded. It is of great significance to the application of arrays in multiply-accumulate computation.

Multiply-accumulate calculations using a memristor crossbar array is an important method to realize neuromorphic computing. However, the memristor array fabrication technology is still immature, and it is difficult to fabricate large-scale arrays with high-yield, which restricts the development of memristor-based neuromorphic computing technology. Therefore, cascading small-scale arrays to achieve the neuromorphic computational ability that can be achieved by large-scale arrays, which is of great significance for promoting the application of memristor-based neuromorphic computing. To address this issue, we present a memristor-based cascaded framework with some basic computation units, several neural network processing units can be cascaded by this means to improve the processing capability of the dataset. Besides, we introduce a split method to reduce pressure of input terminal. Compared with VGGNet and GoogLeNet, the proposed cascaded framework can achieve 93.54% Fashion-MNIST accuracy under the 4.15M parameters. Extensive experiments with Ti/AlOx/TaOx/Pt we fabricated are conducted to show that the circuit simulation results can still provide a high recognition accuracy, and the recognition accuracy loss after circuit simulation can be controlled at around 0.26%.

Recent progress in variational inference has paid much attention to the flexibility of variational posteriors. One promising direction is to use implicit distributions, i.e., distributions without tractable densities as the variational posterior. However, existing methods on implicit posteriors still face challenges of noisy estimation and computational infeasibility when applied to models with high-dimensional latent variables. In this paper, we present a new approach named Kernel Implicit Variational Inference that addresses these challenges. As far as we know, for the first time implicit variational inference is successfully applied to Bayesian neural networks, which shows promising results on both regression and classification tasks.

Recently there have been increasing interests in learning and inference with implicit distributions (i.e., distributions without tractable densities). To this end, we develop a gradient estimator for implicit distributions based on Stein's identity and a spectral decomposition of kernel operators, where the eigenfunctions are approximated by the Nystr\"om method. Unlike the previous works that only provide estimates at the sample points, our approach directly estimates the gradient function, thus allows for a simple and principled out-of-sample extension. We provide theoretical results on the error bound of the estimator and discuss the bias-variance tradeoff in practice. The effectiveness of our method is demonstrated by applications to gradient-free Hamiltonian Monte Carlo and variational inference with implicit distributions. Finally, we discuss the intuition behind the estimator by drawing connections between the Nystr\"om method and kernel PCA, which indicates that the estimator can automatically adapt to the geometry of the underlying distribution.