1 January 2021 in Computational Intelligence and Neuroscience
Convolutional neural networks (CNNs) are effective models for image classification and recognition. Gradient descent optimization (GD) is the basic algorithm for CNN model optimization. Since GD appeared, a series of improved algorithms have been derived. Among these algorithms, adaptive moment estimation (Adam) has been widely recognized. However, local changes are ignored in Adam to some extent. In this paper, we introduce an adaptive learning rate factor based on current and recent gradients. According to this factor, we can dynamically adjust the learning rate of each independent parameter to adaptively adjust the global convergence process. We use the factor to adjust the learning rate for each parameter. The convergence of the proposed algorithm is proven by using the regret bound approach of the online learning framework. In the experimental section, comparisons are conducted between the proposed algorithm and other existing algorithms, such as AdaGrad, RMSprop, Adam, diffGrad, and AdaHMG, on test functions and the MNIST dataset. The results show that Adam and RMSprop combined with our algorithm can not only find the global minimum faster in the experiment using the test function but also have a better convergence curve and higher test set accuracy in experiments using datasets. Our algorithm is a supplement to the existing gradient descent algorithms, which can be combined with many other existing gradient descent algorithms to improve the efficiency of iteration, speed up the convergence of the cost function, and improve the final recognition rate.
9 March 2023 in Computer Methods and Programs in Biomedicine
24 August 2020 in IEEE Access
To accelerate the scanning speed of magnetic resonance imaging (MRI) and improve the quality of magnetic resonance (MR) image reconstruction, a fast MRI technology based on compressed sensing is proposed. Nesterov's accelerated gradient descent (NAG) algorithm uses Nesterov acceleration to optimize the gradient descent (GD) method. However, this form of acceleration factor uses a fixed iteration curve update and can not adapt to different iteration processes. A generalized Nesterov acceleration concept is proposed. Combining the total variation model, a generalized Nesterov accelerated conjugate gradient based on total variation (GNACG_TV) algorithm is proposed. It extends the acceleration factor in a generalized manner, introducing the Frobenius norm of the objective function as a parameter, so that the acceleration factor is related not only to the number of iterations but also to the iteration process and guarantees the convergence of the iterative process. Experiments on three MR images (abdomen, head, and ankles) at different sampling ratios show that the proposed GNACG_TV algorithm compares favorably with conjugate gradient (CG), conjugate gradient based on total variation (CG_TV), Nesterov accelerated conjugate gradient based on total variation (NACG_TV), and conjugate gradient based on adaptive moment estimation (ADAMCG) algorithms in the MSE, PSNR and SSIM exhibit better performance and robustness in denoising performance for the proposed algorithm. Comparing with the result of qualitative and quantitative analysis, it was concluded that the proposed method can better reconstruct under-sampled MR images than other 4 methods. GNACG_TV can further improve the convergence speed based on Nesterov acceleration and get better reconstruction performance.
13 March 2026 in Journal of Mechanics in Medicine and Biology
24 February 2026 in Physical and Engineering Sciences in Medicine