In this survey we discuss properties of specific methods of approximation that belong to a family of greedy approximation methods (greedy algorithms). It is now well understood that we need to study nonlinear sparse representations in order to significantly increase our ability to process (compress, denoise,etc.) large data sets. Sparse representations of a function are not only a powerful analytic tool but they are utilized in many application areas such as image/signal processing and numerical computation. The key to finding sparse representations is the concept ofm-term approximation of the target function by the elements of a given system of functions (dictionary). The fundamental question is how to construct good methods (algorithms) of approximation. Recent results have established that greedy-type algorithms are suitable methods of nonlinear approximation in bothm-term approximation with regard to bases, andm-term approximation with regard to redundant systems. It turns out that there is one fundamental principle that allows us to build good algorithms, both for arbitrary redundant systems and for very simple well-structured bases, such as the Haar basis. This principle is the use of a greedy step in searching for a new element to be added to a givenm-term approximant.

Recently, a new direction in signal processing — “Compressed Sensing” is being actively developed. A number of authors have pointed out a connection between the Compressed Sensing problem and the problem of estimating the Kolmogorov widths, studied in the seventies and eighties of the last century. In this paper we make the above mentioned connection more precise.