This work proposes a decision tree (DT)-based method for initializing a dendritic neuron model (DNM). Neural networks become larger and larger, thus consuming more and more computing resources. This calls for a strong need to prune neurons that do not contribute much to their network’s output. Pruning those with low contribution may lead to a loss of accuracy of DNM. Our proposed method is novel because 1) it can reduce the number of dendrites in DNM while improving training efficiency without affecting accuracy and 2) it can select proper initialization weight and threshold of neurons. The Adam algorithm is used to train DNM after its initialization with our proposed DT-based method. To verify its effectiveness, we apply it to seven benchmark datasets. The results show that decision-tree-initialized DNM is significantly better than the original DNM, k-nearest neighbor, support vector machine, back-propagation neural network, and DT classification methods. It exhibits the lowest model complexity and highest training speed without losing any accuracy. The interactions among attributes can also be observed in its dendritic neurons.
4 September 2024 in IEEE/CAA Journal of Automatica Sinica
To perform well, deep learning (DL) models have to be trained well. Which optimizer should be adopted? We answer this question by discussing how optimizers have evolved from traditional methods like gradient descent to more advanced techniques to address challenges posed by high-dimensional and non-convex problem space. Ongoing challenges include their hyperparameter sensitivity, balancing between convergence and generalization performance, and improving interpretability of optimization processes. Researchers continue to seek robust, efficient, and universally applicable optimizers to advance the field of DL across various domains.
This work considers an extended flexible job-shop scheduling problem from a semiconductor manufacturing environment. To find its high-quality solution in a reasonable time, a learning-based genetic algorithm (LGA) that incorporates a parallel long short-term memory network-embedded autoencoder model is proposed. In it, genetic algorithm is selected as a main optimizer. A novel autoencoder model is trained offline via end-to-end unsupervised learning without relying on labeled data. This model captures the major linkages among decision variables and generates promising solutions in an informative low-dimensional space, striking a balance between computational efficiency and solution quality. To further improve its search ability, a co-evolving framework is designed, which includes both a network-embedded subpopulation and a regular one. The former focuses on its global search while the latter ensures LGA’s convergence. An information exchange method between the two subpopulations balances global and local search, improving its overall optimization ability. This work conducts various numerical experiments to compare LGA with the CPLEX optimizer, several classical heuristics, and some popular methods. Results show that LGA outperforms its peers in finding high-quality solutions in a reasonable time.
6 July 2023 in IEEE Transactions on Neural Networks and Learning Systems
To construct a strong classifier ensemble, base classifiers should be accurate and diverse. However, there is no uniform standard for the definition and measurement of diversity. This work proposes a learners’ interpretability diversity (LID) to measure the diversity of interpretable machine learners. It then proposes a LID-based classifier ensemble. Such an ensemble concept is novel because: 1) interpretability is used as an important basis for diversity measurement and 2) before its training, the difference between two interpretable base learners can be measured. To verify the proposed method’s effectiveness, we choose a decision-tree-initialized dendritic neuron model (DDNM) as a base learner for ensemble design. We apply it to seven benchmark datasets. The results show that the DDNM ensemble combined with LID obtains superior performance in terms of accuracy and computational efficiency compared to some popular classifier ensembles. A random-forest-initialized dendritic neuron model (RDNM) combined with LID is an outstanding representative of the DDNM ensemble.
Friction and wear between mating surfaces significantly affect the efficiency and performance of mechanical systems. Traditional tribological research relies on post-observation methods, limiting the understanding of dynamic friction behavior. In contrast, in situ monitoring provides real-time insights into evolving friction dynamics. This study employs machine learning to monitor polymer wear performance through friction noise. The predictive accuracy of various machine learning methods, including Extremely Randomized Trees, Gradient-Boosting Decision Trees, AdaBoost, LightGBM, Deep Forest, and Deep Neural Networks, is compared for wear-type classification. Additionally, the LSBoost regression is selected as the optimal method for predicting polymer wear-rates across various temperatures. The results underscore the potential of using friction noise and machine learning for real-time wear monitoring, offering valuable insights for tribological system maintenance and failure prediction.
1 June 2023 in CAAI Transactions on Intelligence Technology
The dendritic neural model (DNM) mimics the non‐linearity of synapses in the human brain to simulate the information processing mechanisms and procedures of neurons. This enhances the understanding of biological nervous systems and the applicability of the model in various fields. However, the existing DNM suffers from high complexity and limited generalisation capability. To address these issues, a DNM pruning method with dendrite layer significance constraints is proposed. This method not only evaluates the significance of dendrite layers but also allocates the significance of a few dendrite layers in the trained model to a few dendrite layers, allowing the removal of low‐significance dendrite layers. The simulation experiments on six UCI datasets demonstrate that our method surpasses existing pruning methods in terms of network size and generalisation performance.