Artificial Neural Network (ANN) design is a complex task because its performance depends on the architecture, the selected transfer function, and the learning algorithm used to train the set of synaptic weights. In this paper we present a methodology that automatically designs an ANN using particle swarm optimization algorithms such as Basic Particle Swarm Optimization (PSO), Second Generation of Particle Swarm Optimization (SGPSO), and a New Model of PSO called NMPSO. The aim of these algorithms is to evolve, at the same time, the three principal components of an ANN: the set of synaptic weights, the connections or architecture, and the transfer functions for each neuron. Eight different fitness functions were proposed to evaluate the fitness of each solution and find the best design. These functions are based on the mean square error (MSE) and the classification error (CER) and implement a strategy to avoid overtraining and to reduce the number of connections in the ANN. In addition, the ANN designed with the proposed methodology is compared with those designed manually using the well-known Back-Propagation and Levenberg-Marquardt Learning Algorithms. Finally, the accuracy of the method is tested with different nonlinear pattern classification problems.
Several meta-heuristic algorithms have been pro posed in the last years for solving a wide range of optimization problems. Cuckoo Search Algorithm (CS) is a novel meta heuristic based on the obligate brood parasitic behaviour of some cuckoo species in combination with the Levy flight behavior of some birds and fruit flies. This algorithm has been applied in a wide range of optimization problems; nonetheless, their promising results suggest its application in the field of artificial neural networks, specially during the adjustment of the synaptic weights. On the other hand, spiking neurons are neural models that try to simulate the behavior of biological neurons when they are excited with an input current (input pattern) during a certain period time. Instead of generating a response in its output every iteration, as classical neurons do, this model generates a response (spikes or spike train) only when the model reaches a specific threshold. This response could be coded into a firing rate and perform a pattern classification task according to the firing rate generated with the input current. To perform a classification task the model ought to exhibit the next behavior: patterns from the same class must generate similar firing rates and patterns from other classes have to generate firing rates sufficiently dissimilar to differentiate among the classes. The model needs of a training phase aimed to adjust their synaptic weights and exhibit the desired behavior. In this paper, we describe how the CS algorithm can be useful to train a spiking neuron to be applied in a pattern classification task. The accuracy of the methodology is tested using several pattern recognition problems.
Artificial bee colony (ABC) algorithm has been used in several optimization problems, including the optimization of synaptic weights from an Artificial Neural Network (ANN). However, this is not enough to generate a robust ANN. For that reason, some authors have proposed methodologies based on so-called metaheuristics that automatically allow designing an ANN, taking into account not only the optimization of the synaptic weights as well as the ANN's architecture, and the transfer function of each neuron. However, those methodologies do not generate a reduced design (synthesis) of the ANN. In this paper, we present an ABC based methodology, that maximizes its accuracy and minimizes the number of connections of an ANN by evolving at the same time the synaptic weights, the ANN's architecture and the transfer functions of each neuron. The methodology is tested with several pattern recognition problems.
In the last years, bio-inspired algorithms have shown their power in different non-linear optimization problems. Due to the efficiency and adaptability of bio-inspired algorithms, in this paper we explore a new way to design an artificial neural network (ANN). For this task, a modified PSO algorithm was used. We do not only study the problem of finding the optimal synaptic weights of an ANN but also its topology and transfer functions. In other words, given a set of inputs and desired patterns, with the proposal we are able to find the best topology, the number of neurons, the transfer function for each neuron, as well as the synaptic weights. This allows to designing an ANN to be used to solve a given problem. The proposal is tested using several non-linear problems.