IDEAS home Printed from https://ideas.repec.org/a/cup/jexpos/v1y2014i01p59-80_00.html
   My bibliography  Save this article

Cross-Sample Comparisons and External Validity

Author

Listed:
  • Krupnikov, Yanna
  • Levine, Adam Seth

Abstract

Experimentation is an increasingly popular method among political scientists. While experiments are highly advantageous for creating internally valid conclusions, they are often criticized for being low on external validity. Critical to questions of external validity are the types of subjects who participate in a given experiment, with scholars typically arguing that samples of adults are more externally valid then student samples. Despite the vociferousness of such arguments, these claims have received little empirical treatment. In this paper we empirically test for key differences between student and adult samples by conducting four parallel experiments on each of the three samples commonly used by political scientists. We find that our student and diverse, national adult sample behave consistently and in line with theoretical predictions once relevant moderators are taken into account. The same is not true for our adult convenience sample.

Suggested Citation

  • Krupnikov, Yanna & Levine, Adam Seth, 2014. "Cross-Sample Comparisons and External Validity," Journal of Experimental Political Science, Cambridge University Press, vol. 1(1), pages 59-80, April.
  • Handle: RePEc:cup:jexpos:v:1:y:2014:i:01:p:59-80_00
    as

    Download full text from publisher

    File URL: https://www.cambridge.org/core/product/identifier/S2052263014000074/type/journal_article
    File Function: link to article abstract page
    Download Restriction: no
    ---><---

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Blaine G. Robbins, 2017. "Status, identity, and ability in the formation of trust," Rationality and Society, , vol. 29(4), pages 408-448, November.
    2. Gordon Pennycook & Adam Bear & Evan T. Collins & David G. Rand, 2020. "The Implied Truth Effect: Attaching Warnings to a Subset of Fake News Headlines Increases Perceived Accuracy of Headlines Without Warnings," Management Science, INFORMS, vol. 66(11), pages 4944-4957, November.
    3. Shari De Baets & Dilek Önkal & Wasim Ahmed, 2022. "Do Risky Scenarios Affect Forecasts of Savings and Expenses?," Forecasting, MDPI, vol. 4(1), pages 1-28, February.
    4. Logan S. Casey & Jesse Chandler & Adam Seth Levine & Andrew Proctor & Dara Z. Strolovitch, 2017. "Intertemporal Differences Among MTurk Workers: Time-Based Sample Variations and Implications for Online Data Collection," SAGE Open, , vol. 7(2), pages 21582440177, June.
    5. Adam Seth Levine & Reuben Kline, 2017. "A new approach for evaluating climate change communication," Climatic Change, Springer, vol. 142(1), pages 301-309, May.
    6. Joël Berger & Andreas Diekmann & Stefan Wehrli, 2024. "Does improved upward social mobility foster frustration and conflict? A large-scale online experiment testing Boudon’s model," Rationality and Society, , vol. 36(2), pages 157-182, May.
    7. Antonio A. Arechar & Simon Gächter & Lucas Molleman, 2018. "Conducting interactive experiments online," Experimental Economics, Springer;Economic Science Association, vol. 21(1), pages 99-131, March.
    8. Amanda Hinnant & Roma Subramanian & Rachel Young, 2016. "User comments on climate stories: impacts of anecdotal vs. scientific evidence," Climatic Change, Springer, vol. 138(3), pages 411-424, October.
    9. Sönke Ehret & Sara M. Constantino & Elke U. Weber & Charles Efferson & Sonja Vogt, 2022. "Group Identities Make Fragile Tipping Points," CESifo Working Paper Series 9737, CESifo.
    10. Bachmann, Kremena & Lot, Andre & Xu, Xiaogeng & Hens, Thorsten, 2023. "Experimental Research on Retirement Decision-Making: Evidence from Replications," Journal of Banking & Finance, Elsevier, vol. 152(C).
    11. Keith Finlay & Charles Stoecker & Scott Cunningham, 2015. "Willingness-To-Accept Pharmaceutical Retail Inconvenience: Evidence from a Contingent Choice Experiment," PLOS ONE, Public Library of Science, vol. 10(5), pages 1-10, May.
    12. Kevin E. Levay & Jeremy Freese & James N. Druckman, 2016. "The Demographic and Political Composition of Mechanical Turk Samples," SAGE Open, , vol. 6(1), pages 21582440166, March.
    13. Chen, Tong & Ma, Ke & Bian, Xuemei & Zheng, Chundong & Devlin, James, 2018. "Is high recovery more effective than expected recovery in addressing service failure? — A moral judgment perspective," Journal of Business Research, Elsevier, vol. 82(C), pages 1-9.
    14. Eugene Y. Chan & Jack Lin, 2022. "Political ideology and psychological reactance: how serious should climate change be?," Climatic Change, Springer, vol. 172(1), pages 1-22, May.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:cup:jexpos:v:1:y:2014:i:01:p:59-80_00. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Kirk Stebbing (email available below). General contact details of provider: https://www.cambridge.org/xps .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.