Algorithms involving kernel functions, such as support vector machine (SVM), have attracted huge attention within the artificial learning communities. The performance of these algorithms is greatly influenced by outliers and the choice of kernel functions. This paper introduces a new version of SVM named Deep Decomposition Neural Network Fuzzy SVM (DDNN-FSVM). To this end, we consider an auto-encoder (AE) deep neural network with three layers: input, hidden, and output. Unusually, the AE’s hidden layer comprises a number of neurons greater than the dimension of the input samples, which guarantees linear data separation. The encoder operator is then introduced into the FSVM’s dual to map the training samples to high-dimension spaces. To learn the support vectors and autoencoder parameters, we introduce the loss function and regularization terms in the FSVM dual. To learn from large-scale data, we decompose the resulting model into three small-dimensional submodels using Lagrangian decomposition. To solve the resulting problems, we use SMO, ISDA, and SCG for optimization problems involving large-scale data. We demonstrate that the optimal values of the three submodels solved in parallel provide a good lower bound for the optimal value of the initial model. In addition, thanks to its use of fuzzy weights, DDNN-FSVM is resistant to outliers. Moreover, DDNN-FSVM simultaneously learns the appropriate kernel function and separation path. We tested DDNN-FSVM on several well-known digital and image datasets and compared it to well-known classifiers on the basis of accuracy, precision, f-measure, g-means, and recall. On average, DDNN-FSVM improved on the performance of the classic FSVM across all datasets and outperformed several well-known classifiers.
The Hopfield Recurrent Neural Network (HRNN) is a single-point descent metaheuristic that uses a single potential solution to explore the search space of optimization problems, whose constraints and objective function are aggregated into a typical energy function. The initial point is usually randomly initialized, then moved by applying operators, characterizing the discrete dynamics of the HRNN, which modify its position or direction. Like all single-point metaheuristics, HRNN has certain drawbacks, such as being more likely to get stuck in local optima or miss global optima due to the use of a single point to explore the search space. Moreover, it is more sensitive to the initial point and operator, which can influence the quality and diversity of solutions. Moreover, it can have difficulty with dynamic or noisy environments, as it can lose track of the optimal region or be misled by random fluctuations. To overcome these shortcomings, this paper introduces a population-based fuzzy version of the HRNN, namely Gaussian Takagi–Sugeno Hopfield Recurrent Neural Network (G-TS-HRNN). For each neuron, the G-TS-HRNN associates an input fuzzy variable of d values, described by an appropriate Gaussian membership function that covers the universe of discourse. To build an instance of G-TS-HRNN(s) of size s, we generate s n-uplets of fuzzy values that present the premise of the Takagi–Sugeno system. The consequents are the differential equations governing the dynamics of the HRNN obtained by replacing each premise fuzzy value with the mean of different Gaussians. The steady points of all the rule premises are aggregated using the fuzzy center of gravity equation, considering the level of activity of each rule. G-TS-HRNN is used to solve the random optimization method based on the support vector model. Compared with HRNN, G-TS-HRNN performs better on well-known data sets.