IDEAS home Printed from https://ideas.repec.org/p/osf/osfxxx/sw6kd.html
   My bibliography  Save this paper

Spotlight on researcher decisions – Infrastructure evaluation, instrumental variables, and first-stage specification screening

Author

Listed:
  • Ankel-Peters, Jörg
  • Vance, Colin
  • Bensch, Gunther

Abstract

This paper revisits the instrumental variable (IV) approach in Lipscomb et al. (2013, 2021) to study the impacts of electrification. We first identify errors in the construction of the dataset, including the modelled IV. Revised estimates on main outcomes and mechanisms are statistically insignificant, with substantially lower effect sizes. We second develop a framework that accounts for weak IVs and discourages specification screening. We find that most theoretically justified specifications in the Lipscomb et al. (2013, 2021) data deliver insignificant results. The proposed framework is transferable to other IV applications to reduce potential bias stemming from researcher’s or replicator’s discretion.

Suggested Citation

  • Ankel-Peters, Jörg & Vance, Colin & Bensch, Gunther, 2022. "Spotlight on researcher decisions – Infrastructure evaluation, instrumental variables, and first-stage specification screening," OSF Preprints sw6kd, Center for Open Science.
    • Handle: RePEc:osf:osfxxx:sw6kd
    DOI: 10.31219/osf.io/sw6kd
    as

    Download full text from publisher

    File URL: https://osf.io/download/636d1f2107a7d404fcb95d48/
    Download Restriction: no
    File URL: https://libkey.io/10.31219/osf.io/sw6kd?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Anthony Heyes & Soodeh Saberian, 2022. "Correction to "Temperature and Decisions: Evidence from 207,000 Court Cases" and Reply to Spamann," American Economic Journal: Applied Economics, American Economic Association, vol. 14(4), pages 529-533, October.
    2. David S. Lee & Justin McCrary & Marcelo J. Moreira & Jack Porter, 2022. "Valid t-Ratio Inference for IV," American Economic Review, American Economic Association, vol. 112(10), pages 3260-3290, October.
    3. Garret Christensen & Edward Miguel, 2018. "Transparency, Reproducibility, and the Credibility of Economics Research," Journal of Economic Literature, American Economic Association, vol. 56(3), pages 920-980, September.
    4. Hall, Alastair R & Rudebusch, Glenn D & Wilcox, David W, 1996. "Judging Instrument Relevance in Instrumental Variables Estimation," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 37(2), pages 283-298, May.
    5. Taryn Dinkelman, 2011. "The Effects of Rural Electrification on Employment: New Evidence from South Africa," American Economic Review, American Economic Association, vol. 101(7), pages 3078-3108, December.
    6. Kenneth Lee & Edward Miguel & Catherine Wolfram, 2020. "Experimental Evidence on the Economics of Rural Electrification," Journal of Political Economy, University of Chicago Press, vol. 128(4), pages 1523-1565.
    7. Kruger, Diana I., 2007. "Coffee production effects on child labor and schooling in rural Brazil," Journal of Development Economics, Elsevier, vol. 82(2), pages 448-463, March.
    8. Angrist, Joshua & Kolesár, Michal, 2024. "One instrument to rule them all: The bias and coverage of just-ID IV," Journal of Econometrics, Elsevier, vol. 240(2).
    9. Molly Lipscomb & A. Mushfiq Mobarak & Tania Barham, 2013. "Development Effects of Electrification: Evidence from the Topographic Placement of Hydropower Plants in Brazil," American Economic Journal: Applied Economics, American Economic Association, vol. 5(2), pages 200-231, April.
    10. Abel Brodeur & Nikolai Cook & Anthony Heyes, 2020. "Methods Matter: p-Hacking and Publication Bias in Causal Analysis in Economics," American Economic Review, American Economic Association, vol. 110(11), pages 3634-3660, November.
    11. Douglas Staiger & James H. Stock, 1997. "Instrumental Variables Regression with Weak Instruments," Econometrica, Econometric Society, vol. 65(3), pages 557-586, May.
    12. Isaiah Andrews & James H. Stock & Liyang Sun, 2019. "Weak Instruments in Instrumental Variables Regression: Theory and Practice," Annual Review of Economics, Annual Reviews, vol. 11(1), pages 727-753, August.
    13. Burlig, Fiona, 2018. "Improving transparency in observational social science research: A pre-analysis plan approach," Economics Letters, Elsevier, vol. 168(C), pages 56-60.
    14. Gary Solon & Steven J. Haider & Jeffrey M. Wooldridge, 2015. "What Are We Weighting For?," Journal of Human Resources, University of Wisconsin Press, vol. 50(2), pages 301-316.
    15. Kenneth Lee & Edward Miguel & Catherine Wolfram, 2020. "Does Household Electrification Supercharge Economic Development?," Journal of Economic Perspectives, American Economic Association, vol. 34(1), pages 122-144, Winter.
    16. Young, Alwyn, 2022. "Consistency without inference: instrumental variables in practical application," LSE Research Online Documents on Economics 115011, London School of Economics and Political Science, LSE Library.
    17. Nick Huntington‐Klein & Andreu Arenas & Emily Beam & Marco Bertoni & Jeffrey R. Bloem & Pralhad Burli & Naibin Chen & Paul Grieco & Godwin Ekpe & Todd Pugatch & Martin Saavedra & Yaniv Stopnitzky, 2021. "The influence of hidden researcher decisions in applied microeconomics," Economic Inquiry, Western Economic Association International, vol. 59(3), pages 944-960, July.
    18. Uri Simonsohn & Joseph P. Simmons & Leif D. Nelson, 2020. "Specification curve analysis," Nature Human Behaviour, Nature, vol. 4(11), pages 1208-1214, November.
    19. Young, Alwyn, 2022. "Consistency without Inference: Instrumental Variables in Practical Application," European Economic Review, Elsevier, vol. 147(C).
    20. Mettetal, Elizabeth, 2019. "Irrigation dams, water and infant mortality: Evidence from South Africa," Journal of Development Economics, Elsevier, vol. 138(C), pages 17-40.
    21. Uri Simonsohn & Joseph P. Simmons & Leif D. Nelson, 2020. "Publisher Correction: Specification curve analysis," Nature Human Behaviour, Nature, vol. 4(11), pages 1215-1215, November.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Bensch, Gunther & Ankel-Peters, Jörg & Vance, Colin, 2023. "Spotlight on Researcher Decisions – Infrastructure Evaluation, Instrumental Variables, and Specification Screening," VfS Annual Conference 2023 (Regensburg): Growth and the "sociale Frage" 277703, Verein für Socialpolitik / German Economic Association.
    2. Bensch, Gunther & Gotz, Gunnar & Peters, Jörg, 2020. "Effects of rural electrification on employment: A comment on Dinkelman (2011)," Ruhr Economic Papers 840, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    3. Keane, Michael & Neal, Timothy, 2023. "Instrument strength in IV estimation and inference: A guide to theory and practice," Journal of Econometrics, Elsevier, vol. 235(2), pages 1625-1653.
    4. Angrist, Joshua & Kolesár, Michal, 2024. "One instrument to rule them all: The bias and coverage of just-ID IV," Journal of Econometrics, Elsevier, vol. 240(2).
    5. Ankel-Peters, Jörg & Schmidt, Christoph M., 2023. "Rural electrification, the credibility revolution, and the limits of evidence-based policy," Ruhr Economic Papers 1051, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    6. Dreber, Anna & Johannesson, Magnus, 2023. "A framework for evaluating reproducibility and replicability in economics," Ruhr Economic Papers 1055, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    7. Slichter, David & Tran, Nhan, 2023. "Do better journals publish better estimates?," MPRA Paper 118433, University Library of Munich, Germany.
    8. Wang, Wenjie & Zhang, Yichong, 2024. "Wild bootstrap inference for instrumental variables regressions with weak and few clusters," Journal of Econometrics, Elsevier, vol. 241(1).
    9. Guillaume Coqueret, 2023. "Forking paths in financial economics," Papers 2401.08606, arXiv.org.
    10. Rose, Julian & Neubauer, Florian & Ankel-Peters, Jörg, 2024. "Long-term effects of the targeting the ultra-poor program: A reproducibility and replicability assessment of Banerjee et al. (2021)," Ruhr Economic Papers 1107, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    11. Gupta, Ridhima & Pelli, Martino, 2021. "Electrification and cooking fuel choice in rural India," World Development, Elsevier, vol. 146(C).
    12. Khanna, Shefali & Rowe, Kevin, 2024. "The long-run value of electricity reliability in India," Resource and Energy Economics, Elsevier, vol. 77(C).
    13. Jens Klooster & Mikhail Zhelonkin, 2024. "Outlier robust inference in the instrumental variable model with applications to causal effects," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 39(1), pages 86-106, January.
    14. Suchita Srinivasan, 2023. "Social Policies and Adaptation to Extreme Weather: Evidence from South Africa," CER-ETH Economics working paper series 23/381, CER-ETH - Center of Economic Research (CER-ETH) at ETH Zurich.
    15. Zhaonan Qu & Yongchan Kwon, 2024. "Distributionally Robust Instrumental Variables Estimation," Papers 2410.15634, arXiv.org.
    16. Ngawang Dendup, 2021. "Returns to Grid Electricity on Firewood Consumption and Mechanism," Working Papers 2109, Waseda University, Faculty of Political Science and Economics.
    17. Rose, Julian & Neubauer, Florian & Ankel-Peters, Jörg, 2024. "Long-term effects of the targeting the ultra-poor program: A reproducibility and replicability assessment of Banerjee et al. (2021)," Ruhr Economic Papers 1107, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    18. Fetter, T. Robert & Usmani, Faraz, 2024. "Fracking, farmers, and rural electrification in India," Journal of Development Economics, Elsevier, vol. 170(C).
    19. Dendup, Ngawang, 2022. "Returns to grid electricity on firewood and kerosene: Mechanism," Journal of Environmental Economics and Management, Elsevier, vol. 111(C).
    20. David Boisclair & Roxane Borgès Da Silva & Nathalie de Marcellis-Warin & Pierre-Carl Michaud & Ingrid Peignier, 2022. "Combien de personnes ont développé des symptômes ou contracté la Covid-19 au Québec ? Une étude exploratoire," CIRANO Working Papers 2022s-05, CIRANO.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:osf:osfxxx:sw6kd. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: OSF (email available below). General contact details of provider: https://osf.io/preprints/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.