In this paper, we discuss the convex optimization problem over the fixed point set of a nonexpansive mapping. The main objective of the paper is to accelerate the hybrid steepest descent method for the problem. To this goal, we present a new iterative scheme that utilizes the conjugate gradient direction. Its convergence to the solution is guaranteed under certain assumptions. In order to demonstrate the effectiveness, performance, and convergence of our proposed algorithm, we present numerical comparisons of the algorithm with the existing algorithm.

This article deals with nonconvex stochastic optimization problems in deep learning. Appropriate learning rates, based on theory, for adaptive-learning-rate optimization algorithms (e.g., Adam and AMSGrad) to approximate the stationary points of such problems are provided. These rates are shown to allow faster convergence than previously reported for these algorithms. Specifically, the algorithms are examined in numerical experiments on text and image classification and are shown in experiments to perform better with constant learning rates than algorithms using diminishing learning rates.

In this article, we consider an equilibrium problem: find a point u∈C such that f(u, y) ≥ 0 for all y∈C, where a continuous function satisfies f(x, x) = 0 for all and is a closed convex set. The existing computational methods for this problem require repetitive use of the metric projection onto C, which is often hard to compute. To relax the computational difficulty caused by the metric projection, we present a way to use any firmly nonexpansive mapping T satisfying in place of the metric projection onto C. The proposed method can be applied soundly to the Nash equilibrium problem in noncooperative games.