Access

You are not currently logged in.

Access your personal account or get JSTOR access through your library or other institution:

login

Log in to your personal account or through your institution.

How Much Should We Trust Differences-in-Differences Estimates?

Marianne Bertrand, Esther Duflo and Sendhil Mullainathan
The Quarterly Journal of Economics
Vol. 119, No. 1 (Feb., 2004), pp. 249-275
Published by: Oxford University Press
Stable URL: http://www.jstor.org/stable/25098683
Page Count: 27
  • Download ($42.00)
  • Subscribe ($19.50)
  • Cite this Item
How Much Should We Trust Differences-in-Differences Estimates?
Preview not available

Abstract

Most papers that employ Differences-in-Differences estimation (DD) use many years of data and focus on serially correlated outcomes but ignore that the resulting standard errors are inconsistent. To illustrate the severity of this issue, we randomly generate placebo laws in state-level data on female wages from the Current Population Survey. For each law, we use OLS to compute the DD estimate of its "effect" as well as the standard error of this estimate. These conventional DD standard errors severely understate the standard deviation of the estimators: we find an "effect" significant at the 5 percent level for up to 45 percent of the placebo interventions. We use Monte Carlo simulations to investigate how well existing methods help solve this problem. Econometric corrections that place a specific parametric form on the time-series process do not perform well. Bootstrap (taking into account the autocorrelation of the data) works well when the number of states is large enough. Two corrections based on asymptotic approximation of the variance-covariance matrix work well for moderate numbers of states and one correction that collapses the time series information into a "pre"- and "post"-period and explicitly takes into account the effective sample size works well even for small numbers of states.

Page Thumbnails

  • Thumbnail: Page 
249
    249
  • Thumbnail: Page 
250
    250
  • Thumbnail: Page 
251
    251
  • Thumbnail: Page 
252
    252
  • Thumbnail: Page 
253
    253
  • Thumbnail: Page 
254
    254
  • Thumbnail: Page 
255
    255
  • Thumbnail: Page 
256
    256
  • Thumbnail: Page 
257
    257
  • Thumbnail: Page 
258
    258
  • Thumbnail: Page 
259
    259
  • Thumbnail: Page 
260
    260
  • Thumbnail: Page 
261
    261
  • Thumbnail: Page 
262
    262
  • Thumbnail: Page 
263
    263
  • Thumbnail: Page 
264
    264
  • Thumbnail: Page 
265
    265
  • Thumbnail: Page 
266
    266
  • Thumbnail: Page 
267
    267
  • Thumbnail: Page 
268
    268
  • Thumbnail: Page 
269
    269
  • Thumbnail: Page 
270
    270
  • Thumbnail: Page 
271
    271
  • Thumbnail: Page 
272
    272
  • Thumbnail: Page 
273
    273
  • Thumbnail: Page 
274
    274
  • Thumbnail: Page 
275
    275