We derive the convergence rate of the moving least-squares learning algorithm for regression under the assumption that the samples are drawn from a non-identical sequence of probability measures. The error analysis is carried out by analysing the drift error and using the probability inequalities for the non-identical sampling. When the sequence of marginal distributions converges exponentially to marginal distribution in the dual of a Hölder space, we obtain the satisfactory capacity dependent error bounds of the algorithm that can be arbitrarily close to the rate $O\left(m^{-1}\right)$.

Many learning algorithms use hypothesis spaces which are trained from samples, but little theoretical work has been devoted to the study of these algorithms. In this paper we show that mathematical analysis for the kernel-based coefficient least squares for regression with lq-regularizer, 1≤q≤2, which is essentially different from that for algorithms with hypothesis spaces independent of the sample or depending only on the sample size. The error analysis was carried out under the assumption that the samples are drawn from a non-identical sequence of probability measures and satisfy the β-mixing condition. We use the drift error analysis and the independent-blocks technique to deal with the non-identical and dependent setting, respectively. When the sequence of marginal distributions converges exponentially fast in the dual of a Hölder space and the sampling process satisfies polynomially β-mixing, we obtain the capacity dependent error bounds of the algorithm. As a byproduct, we derive a significantly faster learning rate that can be arbitrarily close to the best rate O(m-1) for the independent and identical samples.

We investigate the regression problem in supervised learning by means of the weak rescaled pure greedy algorithm (WRPGA). We construct learning estimator by applying the WRPGA and deduce the tight upper bounds of the K-functional error estimate for the corresponding greedy learning algorithms in Hilbert spaces. Satisfactory learning rates are obtained under two prior assumptions on the regression function. The application of the WRPGA in supervised learning considerably reduces the computational cost while maintaining its powerful generalization capability when compared with other greedy learning algorithms.

We consider the moving least-square (MLS) method by the coefficient-based regression framework with $l^{q}$ -regularizer $(1\leq q\leq2)$ and the sample dependent hypothesis spaces. The data dependent characteristic of the new algorithm provides flexibility and adaptivity for MLS. We carry out a rigorous error analysis by using the stepping stone technique in the error decomposition. The concentration technique with the $l^{2}$ -empirical covering number is also employed in our study to improve the sample error. We derive the satisfactory learning rate that can be arbitrarily close to the best rate $O(m^{-1})$ under more natural and much simpler conditions.

We consider the regression learning based on the moving least-squares (MLS) framework for the polynomially β-mixing samples. The rigorous error analysis is carried out by using the independent-blocks technique in the sample error estimates. We derive the satisfactory learning rate that can be arbitrarily close to the best rate $O\left(m^{-1}\right)$, when the sequence of marginal distributions converges exponentially to some marginal distribution in the dual of a Hölder space.

In this paper, we investigated the coefficient-based regularized distribution regression for data generated by unbounded sampling processes. The algorithm adopts a two-stage sampling framework: the first-stage sample consists of probability distributions, from which the second-stage sample is drawn. A rigorous capacity-dependent convergence analysis was conducted under more general conditions, and its performance was comparable to that of one-stage sampling learning. Regularization was imposed on the coefficients and the kernel $ K $ was permitted to be indefinite. The important feature of this algorithm is that it can improve the saturation effect suffered by classical kernel ridge regression (KRR). Notably, the output sample values were assumed to satisfy a moment condition (rather than the stricter uniform boundedness constraint common in related works). We derived the convergence error bounds via the novel integral operator techniques, and further established the mini-max optimal learning rates of the algorithm, which were comparable to those achieved under bounded sampling settings.