Activation functions facilitate deep neural networks by introducing non-linearity to the learning process. The non-linearity feature gives the neural network the ability to learn complex patterns. Recently, the most widely used activation function is the Rectified Linear Unit (ReLU). Though, other various existing activation including hand-designed alternatives to ReLU have been proposed. However, none has succeeded in replacing ReLU due to their existing inconsistencies. In this work, activation function called ReLU-Memristor-like Activation Function (RMAF) is proposed to leverage benefits of negative values in neural networks. RMAF introduces a constant parameter (α) and a threshold parameter (p) making the function smooth, non-monotonous, and introduces non-linearity in the network. Our experiments show that, the RMAF works better than ReLU and other activation functions on deeper models and across number of challenging datasets. Firstly, experiments are performed by training and classifying on multilayer perceptron (MLP) over benchmark data such as the Wisconsin breast cancer, MNIST, Iris and Car evaluation. RMAF achieves high performance of 98.74%, 99.67%, 98.81% and 99.42% respectively, compared to Sigmoid, Tanh and ReLU. Secondly, experiments were performed on convolution neural network (ResNet) over MNIST, CIFAR-10 and CIFAR-100 data and observed the proposed activation function achieves higher performance accuracy of 99.73%, 98.77% and 79.82% respectively than Tanh, ReLU and Swish. Additionally, we experimented our work on deep networks i.e. squeeze network (SqueezeNet), Dense connected neural network (DenseNet121) and ImageNet dataset, which RMAF produced the best performance. We note that, the RMAF converges faster than the other functions and can replace ReLU in any neural network due to the efficiency, scalability and its similarity to both ReLU and Swish.

This article investigates the problem of robust exponential stability of fuzzy switched memristive inertial neural networks (FSMINNs) with time-varying delays on mode-dependent destabilizing impulsive control protocol. The memristive model presented here is treated as a switched system rather than employing the theory of differential inclusion and set-value map. To optimize the robust exponentially stable process and reduce the cost of time, hybrid mode-dependent destabilizing impulsive and adaptive feedback controllers are simultaneously applied to stabilize FSMINNs. In the new model, the multiple impulsive effects exist between two switched modes, and the multiple switched effects may also occur between two impulsive instants. Based on switched analysis techniques, the Takagi-Sugeno (T-S) fuzzy method, and the average dwell time, extended robust exponential stability conditions are derived. Finally, simulation is provided to illustrate the effectiveness of the results.

This article investigates the heterogeneous impulsive synchronization for T-S fuzzy probabilistic coupled delayed neural networks (CDNNs) with mode-dependent parameters and piecewise membership functions. To begin with, a novel CDNNs model with adjustable coupling strength and probabilistic coupling delays is designed to ensure the accuracy of the CDNNs model. Meanwhile, the generalized isolated node with four types of mismatched parameters, named heterogeneous isolated delayed neural network, is first considered to extend the synchronization problem. Then, the mode-dependent fuzzy rules are introduced to design the novel model, which implies that switching signals and fuzzy processes are interdependent and can share information to communicate. To improve the hybrid controller’s reliability, the mode-dependent impulses are also developed here, in which the impulsive effects with different properties can occur at any moment in the switching interval. The exponential synchronization conditions are derived by means of the method of auxiliary state variables, Lyapunov–Krasovskii functional, average switching dwell period, and mode-dependent average impulsive dwell period. Moreover, the improved mode-dependent piecewise approximated membership functions are proposed to reduce the main results’ conservatism. Finally, a numerical example is provided to illustrate the effectiveness of the main results.

This paper addresses the problem of dynamic pinning synchronization of fuzzy-dependent-switched (Fds) coupled memristive neural networks (CMNNs) with mismatched dimensions on time scales. To begin with, the probabilistic coupling delays, time scales, mismatched dimensions, and Fps rules are considered to design the novel CMNNs to improve the reliability and generalization ability of the model. Then Fds rules and dynamic pinning control (DPC) method are adopted to design the CMNNs, which can effectively promote the information exchange between the switching signals and the fuzzy processes and can improve the utilization of the communication bandwidth between the nodes of CMNNs. Meanwhile, the method of constructing auxiliary state variables is adopted here to deal with the presented model, so that the coupled and isolated systems with different dimensions can realize information exchange and data sharing. This method also provides a solution for researchers by using low-dimensional systems to estimate or synchronize high-dimensional systems. Moreover, by means of Lyapunov-Krasovskii functional, auxiliary orthogonal matrix, and some inequality processing techniques, the conditions of modified function projective synchronization (MFPS) for FdsCMNNs are derived via the DPC on time scales. Finally, two numerical examples are provided to illustrate the effectiveness of the main results.

Matrix operation is easy to be paralleled by hardware, and the memristor network can realize a parallel matrix computing model with in-memory computing. This article proposes a matrix-friendly genetic algorithm (MGA), in which the population is represented by a matrix and the evolution of population is realized by matrix operations. Compared with the performance of a baseline genetic algorithm (GA) on solving the maximum value of the binary function, MGA can converge better and faster. In addition, MGA is more efficient because of its parallelism on matrix operations, and MGA runs 2.5 times faster than the baseline GA when using the NumPy library. Considering the advantages of the memristor in matrix operations, memristor circuits are designed for the deployment of MGA. This deployment method realizes the parallelization and in-memory computing (memristor is both memory and computing unit) of MGA. In order to verify the effectiveness of this deployment, a feature selection experiment of logistic regression (LR) on Sonar datasets is completed. LR with MGA-based feature selection uses 46 fewer features and achieves 11.9% higher accuracy.