We present an integrated algorithm for simultaneous feature selection (FS) and designing of diverse classifiers using a steady state multiobjective genetic programming (GP), which minimizes three objectives: 1) false positives (FPs); 2) false negatives (FNs); and 3) the number of leaf nodes in the tree. Our method divides a c-class problem into c binary classification problems. It evolves c sets of genetic programs to create c ensembles. During mutation operation, our method exploits the fitness as well as unfitness of features, which dynamically change with generations with a view to using a set of highly relevant features with low redundancy. The classifiers of ith class determine the net belongingness of an unknown data point to the ith class using a weighted voting scheme, which makes use of the FP and FN mistakes made on the training data. We test our method on eight microarray and 11 text data sets with diverse number of classes (from 2 to 44), large number of features (from 2000 to 49151), and high feature-to-sample ratio (from 1.03 to 273.1). We compare our method with a bi-objective GP scheme that does not use any FS and rule size reduction strategy. It depicts the effectiveness of the proposed FS and rule size reduction schemes. Furthermore, we compare our method with four classification methods in conjunction with six features selection algorithms and full feature set. Our scheme performs the best for 380 out of 474 combinations of data sets, algorithm and FS method.
22 February 2017 in IEEE Transactions on Systems, Man, and Cybernetics: Systems
The weights of a multilayer perceptron (MLP) may be altered by multiplicative and/or additive noises if it is implemented in hardware. Moreover, if an MLP is implemented using analog circuits, it is prone to stuck-at 0 faults, i.e., link failures. In this paper, we have proposed a methodology for making an MLP robust with respect to link failures, multiplicative noise, and additive noise. This is achieved by penalizing the system error with three regularizing terms. To train the system we use a weighted sum of the following four terms: 1) mean squared error (MSE); 2) l 2 norm of the weight vector; 3) sum of squares of the first order derivatives of MSE with respect to weights; and 4) sum of squares of the second order derivatives of MSE with respect to weights. The proposed approach has been tested on ten regression and ten classification tasks with link failure, multiplicative noise, and additive noise scenarios. Our experimental results demonstrate the effectiveness of the proposed regularization to achieve robust training of an MLP.
We propose a new archive-based steady-state micro genetic algorithm (ASMiGA). In this context, a new archive maintenance strategy is proposed, which maintains a set of nondominated solutions in the archive unless the archive size falls below a minimum allowable size. It makes the archive size adaptive and dynamic. We have proposed a new environmental selection strategy and a new mating selection strategy. The environmental selection strategy reduces the exploration in less probable objective spaces. The mating selection increases searching in more probable search regions by enhancing the exploitation of existing solutions. A new crossover strategy DE-3 is proposed here. ASMiGA is compared with five well-known multiobjective optimization algorithms of different types-generational evolutionary algorithms (SPEA2 and NSGA-II), archive-based hybrid scatter search, decomposition-based evolutionary approach, and archive-based micro genetic algorithm. For comparison purposes, four performance measures (HV, GD, IGD, and GS) are used on 33 test problems, of which seven problems are constrained. The proposed algorithm outperforms the other five algorithms.
9 July 2019 in IEEE Transactions on Evolutionary Computation
The objectives of this work are to investigate the capability of genetic programming to select and extract linearly separable features when the evolutionary process is guided to achieve the same and to propose an integrated system for that. We decompose a c-class problem into c binary classification problems and evolve c sets of binary classifiers employing a steady-state multiobjective genetic programming with three minimizing objectives. Each binary classifier is composed of a binary tree and a linear support vector machine (SVM). The features extracted by the feature nodes and some of the function nodes of the tree are used to train the SVM. The decision made by the SVM is considered the decision of the corresponding classifier. During crossover and mutation, the SVM-weights are used to determine the usefulness of the corresponding nodes. We also use a fitness function based on Golub’s index to select useful features. To discard less frequently used features, we employ unfitness functions for the feature nodes. We compare our method with 34 classification systems using 18 datasets. The performance of the proposed method is found to be better than 432 out of 570, i.e., 75.79% of comparing cases. Our results confirm that the proposed method is capable of achieving our objectives.
18 June 2018 in IEEE Transactions on Fuzzy Systems
Consider a multiobjective robust optimization problem, where a set of weighted decision makers provides their preferences a priori. The preferences are provided either in the objective space or in the decision variable space using fuzzy numbers. To solve this problem, 1) an indicator to measure consensus, 2) an indicator to measure the robustness of the solutions to their degrees of consensus, and 3) a reformulation of the multiobjective robust optimization problem, are required. It is necessary for the reformulated problem to generate robust solutions that also enjoy high degrees of consensus. In this paper, we have addressed these three issues. For this purpose, we have proposed two approaches to define {consensus}. Then, we have extended these approaches to define {robust consensus}, an indicator to measure the robustness of a given solution to its degree of consensus. Though these approaches can be used to define a countless number of measures, we have proposed 12 definitions of consensus, and hence, robust consensus. Furthermore, we have proposed two ways for the reformulation. Experimental results illustrate that the behaviours of the proposed definitions and of the reformulations are consistent with our expectations.