In this paper, we propose a new pruning algorithm to obtain the optimal number of hidden units of a single layer of a fully connected neural network (NN). The technique relies on a global sensitivity analysis of model output. The relevance of the hidden nodes is determined by analysing the Fourier decomposition of the variance of the model output. Each hidden unit is assigned a ratio (the fraction of variance which the unit accounts for) that gives their ranking. This quantitative information therefore leads to a suggestion of the most favorable units to eliminate. Experimental results suggest that the method can be seen as an effective tool available to the user in controlling the complexity in NNs.
19 December 2013 in IEEE Transactions on Neural Networks and Learning Systems
A new algorithm for the selection of input variables of neural network is proposed. This new method, applied after the training stage, ranks the inputs according to their importance in the variance of the model output. The use of a global sensitivity analysis technique, extended Fourier amplitude sensitivity test, gives the total sensitivity index for each variable, which allows for the ranking and the removal of the less relevant inputs. Applied to some benchmarking problems in the field of features selection, the proposed approach shows good agreement in keeping the relevant variables. This new method is a useful tool for removing superfluous inputs and for system identification.
19 January 2006 in Journal of Solar Energy Engineering
In this paper, emphasis is put on the design of a neural network (NN) to model the direct solar irradiance. Since, unfortunately, a neural network is not a statistician-in-a-box, building a NN for a particular problem is a nontrivial task. As a consequence, we argue that in order to properly model the direct solar irradiance, a systematic methodology must be employed. For this purpose, we propose a two-step approach to building the NN model. The first step deals with a probabilistic interpretation of the NN learning by using Bayesian techniques. The Bayesian approach to modeling offers significant advantages over the classical NN learning process. Among others, one can cite (i) automatic complexity control of the NN using all the available data and (ii) selection of the most important input variables. The second step consists of using a new sensitivity analysis-based pruning method in order to infer the optimal NN structure. We show that the combination of the two approaches makes the practical implementation of the Bayesian techniques more reliable.
25 April 2005 in Journal of Solar Energy Engineering
In this paper, emphasis is put on the design of a neural network to model the direct solar irradiance. Since unfortunately a neural network (NN) is not a statistician in-a-box, building a NN for a particular problem is a non trivial task. As a consequence, we argue that in order to properly model the direct solar irradiance, a systematic methodology must be employed. For this purpose, we propose a two-step approach to building the NN model. The first step deals with a probabilistic interpretation of the NN learning by using Bayesian techniques. The Bayesian approach to modelling offers significant advantages over the classical NN learning process. Among others, one can cite a) automatic complexity control of the NN using all the available data b) selection of the most important input variables. The second step consists in using a new sensitivity analysis-based pruning method in order to infer the optimal NN structure. We show that the combination of the two approaches makes the practical implementation of the Bayesian techniques more reliable.