Hierarchical convex optimization concerns two-stage optimization problems: the first stage problem is a convex optimization; the second stage problem is the minimization of a convex function over the solution set of the first stage problem. For the hierarchical convex optimization, the hybrid steepest descent method (HSDM) can be applied, where the solution set of the first stage problem must be expressed as the fixed point set of a certain nonexpansive operator. In this paper, we propose a nonexpansive operator that yields a computationally efficient update when it is plugged into the HSDM. The proposed operator is inspired by the update of the linearized augmented Lagrangian method. It is applicable to characterize the solution set of recent sophisticated convex optimization problems found in the context of inverse problems, where the sum of multiple proximable convex functions involving linear operators must be minimized to incorporate preferable properties into the minimizers. For such a problem formulation, there has not yet been reported any nonexpansive operator that yields an update free from the inversions of linear operators in cases where it is utilized in the HSDM. Unlike previously known nonexpansive operators, the proposed operator yields an inversion-free update in such cases. As an application of the proposed operator plugged into the HSDM, we also present, in the context of the so-called superiorization, an algorithmic solution to a convex optimization problem over the generalized convex feasible set where the intersection of the hard constraints is not necessarily simple.

In this paper, we propose an acceleration technique of the adaptive filtering scheme called adaptive proximal forward-backward splitting method. For accelerating the convergence rate, the proposed method includes a step to shift the current estimate in the direction of the difference between the current and previous estimates based on the Fast Iterative Shrinkage/Thresholding Algorithm (FISTA). The computational complexity for this additional step is fairly low compared to the overall complexity of the algorithm. As an example of the proposed method, we derive an acceleration of the composition of the Adoptively Weighted Soft-Thresholding (AWST) operator and the exponentially weighted adaptive parallel projection. AWST shrinks the estimated filter coefficients to zero for exploiting the sparsity of the system to be estimated and the exponentially weighted adaptive parallel projection algorithm realizes high accuracy by utilizing all available information at each iteration. This accelerated method improves the steady-state mismatch drastically with its con vergence speed as fast as the proportionate affine projection algorithm.

Recovery of certain piecewise continuous signals from noisy observations has been a major challenge in sciences and engineering. In this paper, in a tight-dimensional representation space, we exploit sparsity hidden in a class of possibly discontinuous signals named finite-dimensional piecewise continuous (FPC) signals. More precisely, we propose a tight-dimensional linear transformation which reveals a certain sparsity in discrete samples of the FPC signals. This transformation is designed by exploiting the fact that most of the consecutive samples are contained in special subspaces. Numerical experiments on recovery of piecewise polynomial signals and piecewise sinusoidal signals show the effectiveness of the revealed sparsity.

Common Spatial Pattern (CSP) methods are widely used to extract the brain activity for brain machine interfacing (BMI) based on electroencephalogram (EEG). For each mental task, CSP methods estimate a covariance matrix of EEG signals and adopt the uniform average of the sample covariance matrices over trials. However, the uniform average is sensitive to outliers caused by e.g. unrelated brain activity. In this paper, we propose an improvement of the estimated covariance matrix utilized in CSP methods by reducing the influence of the outliers as well as guaranteeing positive definiteness. More precisely, our estimation is the projection of the uniform average onto the intersection of two convex sets: the first set is a special reduced dimensional subspace which alleviates the influence of the outliers; the second is the positive definite cone. A numerical experiment supports the effectiveness of the proposed technique.

Many inverse problems in science and engineering are formulated as recovery of piecewise finite-dimensional continuous (PFC) signals. Although the higher-order total variation (HTV) is known to be particularly effective for the sparsity-aware recovery of piecewise polynomials, it remains unclear so far whether the HTV can be extended to other signal models. In this paper, we present a convex regularizer which becomes a generalization of the HTV for the PFC signals. We first design a linear transformation which induces a certain group sparsity of samples of the PFC signals. This linear transformation is designed based on the fact that most of local samples can be interpolated by a fixed linear combination of known basis. Moreover, we provide theoretical evidence that the linear transformed samples have the group sparsity. Then, the proposed regularizer is designed to promote the group sparsity by using the ℓ _1,2 norm. A numerical experiment on recovery of piecewise sinusoidal signals shows the effectiveness of the proposed regularization.