A The learning process of machine learning consists of finding values of unknown weights in a cost function by minimizing the cost function based on learning data. However, since the cost function is not convex, it is conundrum to find the minimum value of the cost function. The existing methods used to find the minimum values usually use the first derivative of the cost function. When even the local minimum (but not a global minimum) is reached, since the first derivative of the cost function becomes zero, the methods give the local minimum values, so that the desired global minimum cannot be found. To overcome this problem, in this paper we modified one of the existing schemes—the adaptive momentum estimation scheme—by adding a new term, so that it can prevent the new optimizer from staying at local minimum. The convergence condition for the proposed scheme and the convergence value are also analyzed, and further explained through several numerical experiments whose cost function is non-convex.
The concept of trend in data and a novel neural network method for the forecasting of upcoming time-series data are proposed in this paper. The proposed method extracts two data sets—the trend and the remainder—resulting in two separate learning sets for training. This method works sufficiently, even when only using a simple recurrent neural network (RNN). The proposed scheme is demonstrated to achieve better performance in selected real-life examples, compared to other averaging-based statistical forecast methods and other recurrent methods, such as long short-term memory (LSTM).
13 June 2024 in Engineering Applications of Artificial Intelligence
In this paper, we introduce a mixed numerical technique for solving fractional differential equations (FDEs) by combining Chebyshev collocation methods and a piecewise quadratic quadrature rule. For getting solutions at each integration step, the fractional integration is calculated in two intervals-all previous time intervals and the current time integration step. The solution at the current integration step is calculated by using Chebyshev interpolating polynomials. To remove a singularity which belongs originally to the FDEs, Lagrangian interpolating technique is considered since the Chebyshev interpolating polynomial can be rewritten as a Lagrangian interpolating form. Moreover, for calculating the fractional integral on the whole previous time intervals, a piecewise quadratic quadrature technique is applied to get higher accuracy. Several numerical experiments demonstrate the efficiency of the proposed method and show numerically convergence orders for both linear and nonlinear cases.
23 November 2018 in Advances in Difference Equations
In this paper, we consider a numerical solution for nonlinear advection–diffusion equation by a backward semi-Lagrangian method. The numerical method is based on the second-order backward differentiation formula for the material derivative and the fourth-order finite difference formula for the diffusion term along the characteristic curve. A modified error correction scheme is newly introduced to efficiently find the departure point of the characteristic curve. Through several numerical simulations, we demonstrate that the proposed method has second and third convergence orders in time and space, respectively, and is efficient and accurate compared to existing techniques. In addition, it is numerically shown that the proposed method has good properties in terms of energy and mass conservation.
1 December 2017 in Journal of Computational Physics