The interval optimization algorithm shows great advantages in bound constrained global optimization. An interval algorithm is presented in this article based on a new selection criterion. The selection criterion is proposed based on numerical experiments and the parameter pf* designed by Casado, Garcia and Csendes in 2000. The proposed criterion at each iteration selects some intervals of which the number is not greater than a constant so that the possible memory problem during the implementation of the algorithm is avoided and the running time of the algorithm is decreased, when the dimension of the problem is increasing. Based on the selection criterion, the proposed algorithm is implemented for a wide set of tested functions which includes easy and hard problems. Numerical experiments show that the proposed algorithm is efficient.
Differential evolution has made great achievements in various fields such as computational sciences, engineering optimization, and operations management in the past decades. It is well known that the control parameter setting plays a very important role in terms of the performance improvement of differential evolution. In this paper, a differential evolution without the scale factor and the crossover probability is presented, which eliminates almost all control parameters except for the population size. The proposed algorithm looks upon each individual as a charged particle to decide on the shift of the individual in the direction of the difference based on the attraction-repulsion mechanism in Coulomb’s Law. Moreover, Taguchi’s parameter design method with the two-level orthogonal array is merged into the crossover operation in order to obtain better individuals in the next generation by means of better combination of factor levels. What is more, a new ratio of the signal-to-noise is proposed for the purpose of fair comparison of the numerical experiment for the tested functions which have an optimal value with 0. Numerical experiments show that the proposed algorithm outperforms the other 5 compared algorithms for the 10 benchmark functions.
Training deep neural networks is challenging. To accelerate training and enhance performance, we propose PadamP, a novel optimization algorithm. PadamP is derived by applying the adaptive estimation of the p-th power of the second-order moments under scale invariance, enhancing projection adaptability by modifying the projection discrimination condition. It is integrated into Adam-type algorithms, accelerating training, boosting performance, and improving generalization in deep learning. Combining projected gradient benefits with adaptive moment estimation, PadamP tackles unconstrained non-convex problems. Convergence for the non-convex case is analyzed, focusing on the decoupling of first-order moment estimation coefficients and second-order moment estimation coefficients. Unlike prior work relying on , our proof generalizes the convergence theorem, enhancing practicality. Experiments using VGG-16 and ResNet-18 on CIFAR-10 and CIFAR-100 show PadamP's effectiveness, with notable performance on CIFAR-10/100, especially for VGG-16. The results demonstrate that PadamP outperforms existing algorithms in terms of convergence speed and generalization ability, making it a valuable addition to the field of deep learning optimization.