Access

You are not currently logged in.

Access JSTOR through your library or other institution:

login

Log in through your institution.

Journal Article

Evaluating Online Labor Markets for Experimental Research: Amazon.com's Mechanical Turk

Adam J. Berinsky, Gregory A. Huber and Gabriel S. Lenz
Political Analysis
Vol. 20, No. 3 (Summer 2012), pp. 351-368
Stable URL: http://www.jstor.org/stable/23260322
Page Count: 18
Were these topics helpful?
See somethings inaccurate? Let us know!

Select the topics that are inaccurate.

Cancel
  • Download ($42.00)
  • Add to My Lists
  • Cite this Item
Evaluating Online Labor Markets for Experimental Research: Amazon.com's Mechanical Turk
Preview not available

Abstract

We examine the trade-offs associated with using Amazon.com's Mechanical Turk (MTurk) interface for subject recruitment. We first describe MTurk and its promise as a vehicle for performing low-cost and easy-to-field experiments. We then assess the internal and external validity of experiments performed using MTurk, employing a framework that can be used to evaluate other subject pools. We first investigate the characteristics of samples drawn from the MTurk population. We show that respondents recruited in this manner are often more representative of the U.S. population than in-person convenience samples—the modal sample in published experimental political science—but less representative than subjects in Internet-based panels or national probability samples. Finally, we replicate important published experimental work using MTurk samples.

Page Thumbnails

  • Thumbnail: Page 
351
    351
  • Thumbnail: Page 
352
    352
  • Thumbnail: Page 
353
    353
  • Thumbnail: Page 
354
    354
  • Thumbnail: Page 
355
    355
  • Thumbnail: Page 
356
    356
  • Thumbnail: Page 
357
    357
  • Thumbnail: Page 
358
    358
  • Thumbnail: Page 
359
    359
  • Thumbnail: Page 
360
    360
  • Thumbnail: Page 
361
    361
  • Thumbnail: Page 
362
    362
  • Thumbnail: Page 
363
    363
  • Thumbnail: Page 
364
    364
  • Thumbnail: Page 
365
    365
  • Thumbnail: Page 
366
    366
  • Thumbnail: Page 
367
    367
  • Thumbnail: Page 
368
    368