Over recent years, there has been a rapid development of deep learning (DL) in both industry and academia fields. However, finding the optimal hyperparameters of a DL model often needs high computational cost and human expertise. To mitigate the above issue, evolutionary computation (EC) as a powerful heuristic search approach has shown significant merits in the automated design of DL models, so-called evolutionary deep learning (EDL). This article aims to analyze EDL from the perspective of automated machine learning (AutoML). Specifically, we first illuminate EDL from DL and EC and regard EDL as an optimization problem. According to the DL pipeline, we systematically introduce EDL methods ranging from data preparation, model generation, to model deployment with a new taxonomy (i.e., what and how to evolve/optimize), and focus on the discussions of solution representation and search paradigm in handling the optimization problem by EC. Finally, key applications, open issues, and potentially promising lines of future research are suggested. This survey has reviewed recent developments of EDL and offers insightful guidelines for the development of EDL.
13 September 2023 in IEEE Transactions on Evolutionary Computation
In multi-objective evolutionary neural architecture search (NAS), existing predictor-based methods commonly suffer from the rank disorder issue that a candidate high-performance architecture may have a poor ranking compared with the worse architecture in terms of the trained predictor.To alleviate the above issue, we aim to train a Pareto-wise end-to-end ranking classifier to simplify the architecture search process by transforming the complex multi-objective NAS task into a simple classification task. To this end, a classifier-based Pareto evolution approach is proposed, where an online classifier is trained to directly predict the dominance relationship between the candidate and reference architectures. Besides, an adaptive clustering method is designed to select reference architectures for the classifier, and an α-domination assisted approach is developed to address the imbalance issue of positive and negative samples. The proposed approach is compared with a number of state-of-the-art NAS methods on widely-used test datasets, and computation results show that the proposed approach is able to alleviate the rank disorder issue and outperforms other methods. Especially, the proposed method is able to find a set of promising network architectures with different model sizes ranging from 2M to 5M under diverse objectives and constraints.
25 March 2024 in IEEE Transactions on Fuzzy Systems
Defect recognition is an important task in intelligent manufacturing. Due to the subjectivity of human annotation, the collected defect data usually contains a lot of noise and unpredictable uncertainties, which have a great negative influence on defect recognition. It is a significant challenge to discover an effective defect recognition model with satisfactory uncertainty processing ability. A natural way is to automatically search for an efficient deep model, which can be realized by neural architecture search (NAS). To achieve this, we propose an efficient fuzzy NAS framework for defect recognition, where the searched architecture can effectively handle uncertain information from the given datasets. Specifically, we first design a fuzzy search space and the related encoding strategy for fuzzy NAS. Then, we propose a comparator-based evolutionary search approach, where an online end-to-end comparator is learned to directly determine the selection of candidate architectures from the evolutionary population. The comparator works in an end-to-end way and it transforms the complex ranking problem of evaluating architectures into a simple classification task, which overcomes the rank disorder issue suffered from traditional performance predictors. A series of experimental results demonstrate that the architecture with fewer #Params (1.22 M) search by fuzzy neural architecture search framework for defect recognition method achieves higher accuracy (92.26%) compared to the state-of-the-art results (i.e., DARTS-PV) on the ELPV dataset, as well as competitive results (accuracy = 76.4%, #Params = 1.04 M) on the CODEBRIM dataset. Experimental results show the effectiveness and efficiency of our proposed method in handling uncertain problems.
Defect recognition is an essential aspect of intelligent manufacturing, but it is a challenging task with noise and unpredictable uncertainties, where convolutional neural networks (CNNs) struggle to achieve good performance. The fuzzy neural network (FNN) emerges as a promising approach to handle uncertainties. However, conventional methods for designing FNNs are tedious and error-prone. A solution is to automatically search for efficient FNN, which can be achieved by neural architecture search (NAS). To achieve NAS for FNN, we propose an efficient classifier-assisted evolutionary multiobjective FNN framework for defect recognition. Considering the characteristics of FNN (e.g., difficult to train and prone to overfitting), we first construct the architecture search as a constrained multiobjective optimization problem, the network accuracy and the architecture size are two conflicting objectives, and the constraint is used to filter out low-quality architectures. Then, we design the search space to incorporate the fuzzy module and develop the corresponding architectural representation and evolutionary operators. Furthermore, the complex regression task of performance evaluation is transformed into a classification task, and a classifier is designed to simplify the performance evaluation process. Massive experiments on four defect recognition datasets (i.e., ELPV, CODEBRIM, MIXEDWM38, and WM-811K) show that the architectures can effectively handle inherent uncertainties from datasets. Our method achieves 94.77% accuracy on ELPV, 81.82% accuracy on CODEBRIM, 98.99% accuracy on MIXEDWM38, and 98.22% accuracy on WM-811K, respectively.
3 April 2026 in IEEE Transactions on Evolutionary Computation
Neural architecture search (NAS) faces a significant challenge due to the high computational cost of architecture evaluation. Zero-shot (ZS) proxies have been proposed as an efficient alternative to reduce evaluation overhead; however, their limited rank correlation often leads to sub-optimal architecture selection. To alleviate this issue, recent studies attempt to combine multiple ZS proxies, yet most existing approaches rely on heuristic aggregation guided by domain knowledge and remain restricted to a single evaluation scenario, which limits both effectiveness and interpretability. In this paper, we propose a novel perspective that treats ZS proxies as architectural features and formulates their combination as an evolutionary symbolic learning problem. Specifically, we leverage a genetic programming based symbolic learning framework to automatically evolve interpretable proxy combinations, enabling flexible exploration of symbolic structures without manual design. The evolved symbolic models capture complex and non-linear relationships between ZS proxies and architectural performance while maintaining strong interpretability. Furthermore, we systematically model three evaluation scenarios, including pointwise, pairwise, and listwise settings, and design scenario-specific fitness functions to guide evolutionary search toward different ranking objectives. Extensive experiments across 19 tasks using 13 representative ZS proxies are conducted to evaluate the proposed framework. The results demonstrate that the evolved symbolic models consistently achieve improved rank correlation and provide valuable insights into proxy interaction patterns under different evaluation scenarios, highlighting the effectiveness and generalizability of the proposed approach for NAS.