IDEAS home Printed from https://ideas.repec.org/a/bpj/causin/v9y2021i1p264-284n5.html
   My bibliography  Save this article

Conditional as-if analyses in randomized experiments

Author

Listed:
  • Pashley Nicole E.

    (Department of Statistics, Rutgers University, Piscataway, NJ 08854, United States of America)

  • Basse Guillaume W.

    (Department of Statistics, Stanford University, Stanford, CA 94305, United States of America)

  • Miratrix Luke W.

    (Harvard Graduate School of Education, Cambridge, MA 02138, United States of America)

Abstract

The injunction to “analyze the way you randomize” is well known to statisticians since Fisher advocated for randomization as the basis of inference. Yet even those convinced by the merits of randomization-based inference seldom follow this injunction to the letter. Bernoulli randomized experiments are often analyzed as completely randomized experiments, and completely randomized experiments are analyzed as if they had been stratified; more generally, it is not uncommon to analyze an experiment as if it had been randomized differently. This article examines the theoretical foundation behind this practice within a randomization-based framework. Specifically, we ask when is it legitimate to analyze an experiment randomized according to one design as if it had been randomized according to some other design. We show that a sufficient condition for this type of analysis to be valid is that the design used for analysis should be derived from the original design by an appropriate form of conditioning. We use our theory to justify certain existing methods, question others, and finally suggest new methodological insights such as conditioning on approximate covariate balance.

Suggested Citation

  • Pashley Nicole E. & Basse Guillaume W. & Miratrix Luke W., 2021. "Conditional as-if analyses in randomized experiments," Journal of Causal Inference, De Gruyter, vol. 9(1), pages 264-284, January.
    • Handle: RePEc:bpj:causin:v:9:y:2021:i:1:p:264-284:n:5
    DOI: 10.1515/jci-2021-0012
    as

    Download full text from publisher

    File URL: https://doi.org/10.1515/jci-2021-0012
    Download Restriction: no
    File URL: https://libkey.io/10.1515/jci-2021-0012?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Rolf Sundberg, 2003. "Conditional statistical inference and quantification of relevance," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 65(1), pages 299-315, February.
    2. Xinran Li & Peng Ding, 2017. "General Forms of Finite Population Central Limit Theorems with Applications to Causal Inference," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(520), pages 1759-1769, October.
    3. Alberto Abadie & Guido W. Imbens, 2012. "A Martingale Representation for Matching Estimators," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 107(498), pages 833-843, June.
    4. D. R. Cox, 2009. "Randomization in the Design of Experiments," International Statistical Review, International Statistical Institute, vol. 77(3), pages 415-429, December.
    5. Ho, Daniel E. & Imai, Kosuke & King, Gary & Stuart, Elizabeth A., 2007. "Matching as Nonparametric Preprocessing for Reducing Model Dependence in Parametric Causal Inference," Political Analysis, Cambridge University Press, vol. 15(3), pages 199-236, July.
    6. Alberto Abadie & Guido W. Imbens, 2006. "Large Sample Properties of Matching Estimators for Average Treatment Effects," Econometrica, Econometric Society, vol. 74(1), pages 235-267, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Glazer Amanda K. & Pimentel Samuel D., 2023. "Robust inference for matching under rolling enrollment," Journal of Causal Inference, De Gruyter, vol. 11(1), pages 1-19, January.
    2. Fredrik Savje, 2019. "On the inconsistency of matching without replacement," Papers 1907.07288, arXiv.org, revised Jun 2021.
    3. Zhexiao Lin & Peng Ding & Fang Han, 2021. "Estimation based on nearest neighbor matching: from density ratio to average treatment effect," Papers 2112.13506, arXiv.org.
    4. Magambo, Isaiah & Dikgang, Johane & Gelo, Dambala & Tregenna, Fiona, 2021. "Gold-Mining Pollution Exposure, Health Effects and Private Healthcare Expenditure in Tanzania," MPRA Paper 108800, University Library of Munich, Germany.
    5. Zichen Deng & Maarten Lindeboom, 2021. "Early-life Famine Exposure, Hunger Recall and Later-life Health," Tinbergen Institute Discussion Papers 21-054/V, Tinbergen Institute.
    6. Mansaray, Alhassan & Coleman, Simeon & Ataullah, Ali & Sirichand, Kavita, 2021. "Residual government ownership in public-private partnership projects," Journal of Government and Economics, Elsevier, vol. 4(C).
    7. Jasjeet Singh Sekhon & Richard D. Grieve, 2012. "A matching method for improving covariate balance in cost‐effectiveness analyses," Health Economics, John Wiley & Sons, Ltd., vol. 21(6), pages 695-714, June.
    8. Blackman, Allen & Goff, Leonard & Rivera Planter, Marisol, 2018. "Does eco-certification stem tropical deforestation? Forest Stewardship Council certification in Mexico," Journal of Environmental Economics and Management, Elsevier, vol. 89(C), pages 306-333.
    9. Estifanos, Tafesse Kefyalew & Polyakov, Maksym & Pandit, Ram & Hailu, Atakelty & Burton, Michael, 2020. "The impact of protected areas on the rural households’ incomes in Ethiopia," Land Use Policy, Elsevier, vol. 91(C).
    10. Yihui He & Fang Han, 2023. "On propensity score matching with a diverging number of matches," Papers 2310.14142, arXiv.org, revised Nov 2023.
    11. Arriagada, Rodrigo & Villaseñor, Adrián & Rubiano, Eliana & Cotacachi, David & Morrison, Judith, 2018. "Analysing the impacts of PES programmes beyond economic rationale: Perceptions of ecosystem services provision associated to the Mexican case," Ecosystem Services, Elsevier, vol. 29(PA), pages 116-127.
    12. Blackman, Allen, 2015. "Strict versus mixed-use protected areas: Guatemala's Maya Biosphere Reserve," Ecological Economics, Elsevier, vol. 112(C), pages 14-24.
    13. Barbopoulos, Leonidas G. & Adra, Samer & Saunders, Anthony, 2020. "Macroeconomic news and acquirer returns in M&As: The impact of investor alertness," Journal of Corporate Finance, Elsevier, vol. 64(C).
    14. Lenis, David & Ackerman, Benjamin & Stuart, Elizabeth A., 2018. "Measuring model misspecification: Application to propensity score methods with complex survey data," Computational Statistics & Data Analysis, Elsevier, vol. 128(C), pages 48-57.
    15. Songliang Chen & Fang Han, 2024. "On the limiting variance of matching estimators," Papers 2411.05758, arXiv.org.
    16. Barbopoulos, Leonidas G. & Adra, Samer, 2016. "The earnout structure matters: Takeover premia and acquirer gains in earnout financed M&As," International Review of Financial Analysis, Elsevier, vol. 45(C), pages 283-294.
    17. Timothy B. Armstrong & Michal Kolesár, 2021. "Finite‐Sample Optimal Estimation and Inference on Average Treatment Effects Under Unconfoundedness," Econometrica, Econometric Society, vol. 89(3), pages 1141-1177, May.
    18. Robert J. Johnston & Klaus Moeltner, 2019. "Special Flood Hazard Effects on Coastal and Interior Home Values: One Size Does Not Fit All," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 74(1), pages 181-210, September.
    19. Jan-Hinrik Meyer-Sahling & Will Lowe & Christian van Stolk, 2016. "Silent professionalization: EU integration and the professional socialization of public officials in Central and Eastern Europe," European Union Politics, , vol. 17(1), pages 162-183, March.
    20. Siddique Abu Bakkar, 2020. "Identity-based Earning Discrimination among Chinese People," IZA Journal of Development and Migration, Sciendo & Forschungsinstitut zur Zukunft der Arbeit GmbH (IZA), vol. 11(1), pages 1-42, January.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bpj:causin:v:9:y:2021:i:1:p:264-284:n:5. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Peter Golla (email available below). General contact details of provider: https://www.degruyter.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.