IDEAS home Printed from https://ideas.repec.org/p/osf/metaar/d5eud_v1.html
   My bibliography  Save this paper

Panel Data and Experimental Design

Author

Listed:
  • Burlig, Fiona
  • Preonas, Louis
  • Woerman, Matt

Abstract

How should researchers design experiments to detect treatment effects with panel data? In this paper, we derive analytical expressions for the variance of panel estimators under non-i.i.d. error structures, which inform power calculations in panel data settings. Using Monte Carlo simulation, we demonstrate that, with correlated errors, traditional methods for experimental design result in experiments that are incorrectly powered with proper inference. Failing to account for serial correlation yields overpowered experiments in short panels and underpowered experiments in long panels. Using both data from a randomized experiment in China and a high-frequency dataset of U.S. electricity consumption, we show that these results hold in real-world settings. Our theoretical results enable us to achieve correctly powered experiments in both simulated and real data. This paper provides researchers with the tools to design well-powered experiments in panel data settings.

Suggested Citation

  • Burlig, Fiona & Preonas, Louis & Woerman, Matt, 2017. "Panel Data and Experimental Design," MetaArXiv d5eud_v1, Center for Open Science.
    • Handle: RePEc:osf:metaar:d5eud_v1
    DOI: 10.31219/osf.io/d5eud_v1
    as

    Download full text from publisher

    File URL: https://osf.io/download/58ba02b46c613b01f53b3292/
    Download Restriction: no
    File URL: https://libkey.io/10.31219/osf.io/d5eud_v1?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Joshua D. Angrist & Jörn-Steffen Pischke, 2010. "The Credibility Revolution in Empirical Economics: How Better Research Design Is Taking the Con out of Econometrics," Journal of Economic Perspectives, American Economic Association, vol. 24(2), pages 3-30, Spring.
    2. Rachel Glennerster & Kudzai Takavarasha, 2013. "Running Randomized Evaluations: A Practical Guide," Economics Books, Princeton University Press, edition 1, number 10085.
    3. Koichiro Ito & Takanori Ida & Makoto Tanaka, 2015. "The Persistence of Moral Suasion and Economic Incentives: Field Experimental Evidence from Energy Demand," NBER Working Papers 20910, National Bureau of Economic Research, Inc.
    4. A. Colin Cameron & Douglas L. Miller, 2015. "A Practitioner’s Guide to Cluster-Robust Inference," Journal of Human Resources, University of Wisconsin Press, vol. 50(2), pages 317-372.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Burlig, Fiona & Preonas, Louis & Woerman, Matt, 2020. "Panel data and experimental design," Journal of Development Economics, Elsevier, vol. 144(C).
    2. Brox, Enzo & Krieger, Tommy, 2022. "Birthplace diversity and team performance," Labour Economics, Elsevier, vol. 79(C).
    3. Heinemann, Friedrich & Moessinger, Marc-Daniel & Yeter, Mustafa, 2018. "Do fiscal rules constrain fiscal policy? A meta-regression-analysis," European Journal of Political Economy, Elsevier, vol. 51(C), pages 69-92.
    4. Maclean, Johanna Catherine & Tello-Trillo, Sebastian & Webber, Douglas, 2023. "Losing insurance and psychiatric hospitalizations," Journal of Economic Behavior & Organization, Elsevier, vol. 205(C), pages 508-527.
    5. Jonas Minet Kinge & Jostein Grytten, 2021. "The impact of primary care physician density on perinatal health: Evidence from a natural experiment," Health Economics, John Wiley & Sons, Ltd., vol. 30(12), pages 2974-2994, December.
    6. d’Albis, Hippolyte & El Mekkaoui, Najat & Legendre, Bérangère, 2023. "Health accidents and wealth decline in old age," Social Science & Medicine, Elsevier, vol. 332(C).
    7. Garret Christensen & Edward Miguel, 2018. "Transparency, Reproducibility, and the Credibility of Economics Research," Journal of Economic Literature, American Economic Association, vol. 56(3), pages 920-980, September.
    8. Bellemare, Marc F. & Masaki, Takaaki & Pepinsky, Thomas B., 2015. "Lagged Explanatory Variables and the Estimation of Causal Effects," MPRA Paper 62350, University Library of Munich, Germany, revised 23 Feb 2015.
    9. Alex Eble & Peter Boone & Diana Elbourne, 2017. "On Minimizing the Risk of Bias in Randomized Controlled Trials in Economics," The World Bank Economic Review, World Bank, vol. 31(3), pages 687-707.
    10. Aleksey Oshchepkov & Anna Shirokanova, 2020. "Multilevel Modeling For Economists: Why, When And How," HSE Working papers WP BRP 233/EC/2020, National Research University Higher School of Economics.
    11. Ariel Ortiz-Bobea, 2021. "Climate, Agriculture and Food," Papers 2105.12044, arXiv.org.
    12. Huang, Xuhui & Zhou, Tao & Zhang, Ning, 2025. "How does the carbon market influence the marginal abatement cost? Evidence from China's coal-fired power plants," Applied Energy, Elsevier, vol. 378(PA).
    13. Dhaval Dave & Monica Deza & Brady Horn, 2021. "Prescription drug monitoring programs, opioid abuse, and crime," Southern Economic Journal, John Wiley & Sons, vol. 87(3), pages 808-848, January.
    14. Kim, Jeong-Bon & Zhang, Eliza Xia & Zhong, Kai, 2021. "Does unionization affect the manager–shareholder conflict? Evidence from firm-specific stock price crash risk," Journal of Corporate Finance, Elsevier, vol. 69(C).
    15. Weber, Sylvain & Puddu, Stefano & Pacheco, Diana, 2017. "Move it! How an electric contest motivates households to shift their load profile," Energy Economics, Elsevier, vol. 68(C), pages 255-270.
    16. Susan Athey & Raj Chetty & Guido Imbens, 2020. "Combining Experimental and Observational Data to Estimate Treatment Effects on Long Term Outcomes," Papers 2006.09676, arXiv.org.
    17. Mark J. McCabe & Frank Mueller-Langer, 2019. "Does Data Disclosure Increase Citations? Empirical Evidence from a Natural Experiment in Leading Economics Journals," JRC Working Papers on Digital Economy 2019-02, Joint Research Centre.
    18. Giovanni Peri & Vasil Yasenov, 2019. "The Labor Market Effects of a Refugee Wave: Synthetic Control Method Meets the Mariel Boatlift," Journal of Human Resources, University of Wisconsin Press, vol. 54(2), pages 267-309.
    19. Christensen, Garret & Miguel, Edward, 2017. "Transparency, Reproducibility, and the Credibility of Economics Research," MetaArXiv 9a3rw_v1, Center for Open Science.
    20. Maclean, J. Catherine & Tello-Trillo, Sebastian & Webber, Douglas A., 2019. "Losing Insurance and Behavioral Health Hospitalizations: Evidence from a Large-Scale Medicaid Disenrollment," IZA Discussion Papers 12463, Institute of Labor Economics (IZA).

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:osf:metaar:d5eud_v1. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: OSF (email available below). General contact details of provider: https://osf.io/preprints/metaarxiv .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.