If you need an accessible version of this item please contact JSTOR User Support

Improvements on Cross-Validation: The .632+ Bootstrap Method

Bradley Efron and Robert Tibshirani
Journal of the American Statistical Association
Vol. 92, No. 438 (Jun., 1997), pp. 548-560
DOI: 10.2307/2965703
Stable URL: http://www.jstor.org/stable/2965703
Page Count: 13
  • Download PDF
  • Cite this Item

You are not currently logged in.

Access your personal account or get JSTOR access through your library or other institution:

login

Log in to your personal account or through your institution.

If you need an accessible version of this item please contact JSTOR User Support
Improvements on Cross-Validation: The .632+ Bootstrap Method
Preview not available

Abstract

A training set of data has been used to construct a rule for predicting future responses. What is the error rate of this rule? This is an important question both for comparing models and for assessing a final selected model. The traditional answer to this question is given by cross-validation. The cross-validation estimate of prediction error is nearly unbiased but can be highly variable. Here we discuss bootstrap estimates of prediction error, which can be thought of as smoothed versions of cross-validation. We show that a particular bootstrap method, the .632+ rule, substantially outperforms cross-validation in a catalog of 24 simulation experiments. Besides providing point estimates, we also consider estimating the variability of an error rate estimate. All of the results here are nonparametric and apply to any possible prediction rule; however, we study only classification problems with 0-1 loss in detail. Our simulations include "smooth" prediction rules like Fisher's linear discriminant function and unsmooth ones like nearest neighbors.

Page Thumbnails

  • Thumbnail: Page 
548
    548
  • Thumbnail: Page 
549
    549
  • Thumbnail: Page 
550
    550
  • Thumbnail: Page 
551
    551
  • Thumbnail: Page 
552
    552
  • Thumbnail: Page 
553
    553
  • Thumbnail: Page 
554
    554
  • Thumbnail: Page 
555
    555
  • Thumbnail: Page 
556
    556
  • Thumbnail: Page 
557
    557
  • Thumbnail: Page 
558
    558
  • Thumbnail: Page 
559
    559
  • Thumbnail: Page 
560
    560