Peter L. Bartlett

Peter L. Bartlett

Auto-generated profile, based on publicly available information

Peter L. Bartlett

University of California, Berkeley, USA

Research Focus & Metrics

Academic Career Length

35Years (1992-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
1 May 2000 in Neural Computation
Verified
Journal Article
Closed access
BSBernhard Schölkopf
Bernhard Schölkopf
ASAlex J. Smola
Alex J. Smola
RWRobert C. Williamson
Robert C. Williamson
PBPeter L. Bartlett
Peter L. Bartlett
We propose a new class of support vector algorithms for regression and classification. In these algorithms, a parameter ν lets one effectively control the number of support vectors. While this can be useful in its own right, the parameterization has the additional benefit of enabling us to eliminate one of the other free parameters of the algorithm: the accuracy parameter ε in the regression case, and the regularization constant C in the classification case. We describe the algorithms, give some theoretical results concerning the meaning and the choice of ν, and report experimental results.

1 October 1998 in The Annals of Statistics
Verified
Journal Article
Open access
PBPeter Bartlett
Peter Bartlett
YFYoav Freund
Yoav Freund
WLWee Sun Lee
Wee Sun Lee
RSRobert E. Schapire
Robert E. Schapire
One of the surprising recurring phenomena observed in experiments with boosting is that the test error of the generated classifier usually does not increase as its size becomes very large, and often is observed to decrease even after the training error reaches zero. In this paper, we show that this phenomenon is related to the distribution of margins of the training examples with respect to the generated voting classification rule, where the margin of an example is simply the difference between the number of correct votes and the maximum number of votes received by any incorrect label. We show that techniques used in the analysis of Vapnik’s support vector classifiers and of neural networks with small weights can be applied to voting methods to relate the margin distribution to the test error. We also show theoretically and experimentally that boosting is especially effective at increasing the margins of the training examples. Finally, we compare our explanation to those based on the bias-variance decomposition.


1 March 2006 in Journal of the American Statistical Association
Verified
Journal Article
Closed access
PBPeter L Bartlett
Peter L Bartlett
MJMichael I Jordan
Michael I Jordan
JMJon D McAuliffe
Jon D McAuliffe
Many of the classification algorithms developed in the machine learning literature, including the support vector machine and boosting, can be viewed as minimum contrast methods that minimize a conv...

1 March 1998 in IEEE Transactions on Information Theory
Verified
Journal Article
Closed access
PBP.L. Bartlett
P.L. Bartlett
Sample complexity results from computational learning theory, when applied to neural network learning for pattern classification problems, suggest that for good generalization performance the number of training examples should grow at least linearly with the number of adjustable parameters in the network. Results in this paper show that if a large neural network is used for a pattern classification problem and the learning algorithm finds a network with small weights that has small squared error on the training patterns, then the generalization performance depends on the size of the weights rather than the number of weights. For example, consider a two-layer feedforward network of sigmoid units, in which the sum of the magnitudes of the weights associated with each unit is bounded by A and the input dimension is n. We show that the misclassification probability is no more than a certain error estimate (that is related to squared error on the training set) plus A/sup 3/ /spl radic/((log n)/m) (ignoring log A and log m factors), where m is the number of training patterns. This may explain the generalization performance of neural networks, particularly when the number of training examples is considerably smaller than the number of weights. It also supports heuristics (such as weight decay and early stopping) that attempt to keep the weights small during training. The proof techniques appear to be useful for the analysis of other pattern classifiers: when the input domain is a totally bounded metric space, we use the same approach to give upper bounds on misclassification probability for classifiers with decision boundaries that are far from the training examples.

1 August 2005 in The Annals of Statistics
Verified
Journal Article
Open access
PDF file
PBPeter L. Bartlett
Peter L. Bartlett
OBOlivier Bousquet
Olivier Bousquet
SMShahar Mendelson
Shahar Mendelson
We propose new bounds on the error of learning algorithms in terms of a data-dependent notion of complexity. The estimates we establish give optimal rates and are based on a local and empirical version of Rademacher averages, in the sense that the Rademacher averages are computed from the data, on a subset of functions with small empirical error. We present some applications to classification and prediction with convex function classes, and with kernel classes in particular.