IDEAS home Printed from https://ideas.repec.org/a/bla/istatr/v88y2020i1p101-121.html
   My bibliography  Save this article

Sampling‐based Randomised Designs for Causal Inference under the Potential Outcomes Framework

Author

Listed:
  • Zach Branson
  • Tirthankar Dasgupta

Abstract

We establish the inferential properties of the mean‐difference estimator for the average treatment effect in randomised experiments where each unit in a population is randomised to one of two treatments and then units within treatment groups are randomly sampled. The properties of this estimator are well understood in the experimental design scenario where first units are randomly sampled and then treatment is randomly assigned but not for the aforementioned scenario where the sampling and treatment assignment stages are reversed. We find that the inferential properties of the mean‐difference estimator under this experimental design scenario are identical to those under the more common sample‐first‐randomise‐second design. This finding will bring some clarifications about sampling‐based randomised designs for causal inference, particularly for settings where there is a finite super‐population. Finally, we explore to what extent pre‐treatment measurements can be used to improve upon the mean‐difference estimator for this randomise‐first‐sample‐second design. Unfortunately, we find that pre‐treatment measurements are often unhelpful in improving the precision of average treatment effect estimators under this design, unless a large number of pre‐treatment measurements that are highly associative with the post‐treatment measurements can be obtained. We confirm these results using a simulation study based on a real experiment in nanomaterials.

Suggested Citation

  • Zach Branson & Tirthankar Dasgupta, 2020. "Sampling‐based Randomised Designs for Causal Inference under the Potential Outcomes Framework," International Statistical Review, International Statistical Institute, vol. 88(1), pages 101-121, April.
    • Handle: RePEc:bla:istatr:v:88:y:2020:i:1:p:101-121
    DOI: 10.1111/insr.12339
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/insr.12339
    Download Restriction: no
    File URL: https://libkey.io/10.1111/insr.12339?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Middleton, Joel A., 2008. "Bias of the regression estimator for experiments using clustered random assignment," Statistics & Probability Letters, Elsevier, vol. 78(16), pages 2654-2659, November.
    2. Miratrix, Luke W. & Sekhon, Jasjeet S. & Theodoridis, Alexander G. & Campos, Luis F., 2018. "Worth Weighting? How to Think About and Use Weights in Survey Experiments," Political Analysis, Cambridge University Press, vol. 26(3), pages 275-291, July.
    3. Rahul Mukerjee & Tirthankar Dasgupta & Donald B. Rubin, 2018. "Using Standard Tools From Finite Population Sampling to Improve Causal Inference for Complex Experiments," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(522), pages 868-881, April.
    4. Samii, Cyrus & Aronow, Peter M., 2012. "On equivalencies between design-based and regression-based variance estimators for randomized experiments," Statistics & Probability Letters, Elsevier, vol. 82(2), pages 365-370.
    5. Guido W. Imbens, 2004. "Nonparametric Estimation of Average Treatment Effects Under Exogeneity: A Review," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 4-29, February.
    6. Xinran Li & Peng Ding, 2017. "General Forms of Finite Population Central Limit Theorems with Applications to Causal Inference," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(520), pages 1759-1769, October.
    7. Peng Ding & Avi Feller & Luke Miratrix, 2016. "Randomization inference for treatment effect variation," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 78(3), pages 655-671, June.
    8. Coppock, Alexander, 2019. "Generalizing from Survey Experiments Conducted on Mechanical Turk: A Replication Approach," Political Science Research and Methods, Cambridge University Press, vol. 7(3), pages 613-628, July.
    9. Imbens,Guido W. & Rubin,Donald B., 2015. "Causal Inference for Statistics, Social, and Biomedical Sciences," Cambridge Books, Cambridge University Press, number 9780521885881, September.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Fangzhou Su & Peng Ding, 2021. "Model‐assisted analyses of cluster‐randomized experiments," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 83(5), pages 994-1015, November.
    2. Joel A. Middleton, 2021. "Unifying Design-based Inference: On Bounding and Estimating the Variance of any Linear Estimator in any Experimental Design," Papers 2109.09220, arXiv.org.
    3. Nicole E. Pashley & Luke W. Miratrix, 2021. "Insights on Variance Estimation for Blocked and Matched Pairs Designs," Journal of Educational and Behavioral Statistics, , vol. 46(3), pages 271-296, June.
    4. Jiafeng Chen, 2021. "Nonparametric Treatment Effect Identification in School Choice," Papers 2112.03872, arXiv.org, revised Oct 2023.
    5. Haoge Chang, 2023. "Design-based Estimation Theory for Complex Experiments," Papers 2311.06891, arXiv.org.
    6. Ganesh Karapakula, 2023. "Stable Probability Weighting: Large-Sample and Finite-Sample Estimation and Inference Methods for Heterogeneous Causal Effects of Multivalued Treatments Under Limited Overlap," Papers 2301.05703, arXiv.org, revised Jan 2023.
    7. Lupparelli, Monia & Mattei, Alessandra, 2020. "Joint and marginal causal effects for binary non-independent outcomes," Journal of Multivariate Analysis, Elsevier, vol. 178(C).
    8. Caloffi, Annalisa & Freo, Marzia & Ghinoi, Stefano & Mariani, Marco & Rossi, Federica, 2022. "Assessing the effects of a deliberate policy mix: The case of technology and innovation advisory services and innovation vouchers," Research Policy, Elsevier, vol. 51(6).
    9. Clément de Chaisemartin, 2022. "Trading-off Bias and Variance in Stratified Experiments and in Staggered Adoption Designs, Under a Boundedness Condition on the Magnitude of the Treatment Effect," Working Papers hal-03873919, HAL.
    10. Florian Gunsilius & Yuliang Xu, 2021. "Matching for causal effects via multimarginal unbalanced optimal transport," Papers 2112.04398, arXiv.org, revised Jul 2022.
    11. Ashesh Rambachan & Jonathan Roth, 2020. "Design-Based Uncertainty for Quasi-Experiments," Papers 2008.00602, arXiv.org, revised Oct 2024.
    12. Julius Owusu, 2023. "Randomization Inference of Heterogeneous Treatment Effects under Network Interference," Papers 2308.00202, arXiv.org, revised Jan 2024.
    13. Özler, Berk & Çelik, Çiğdem & Cunningham, Scott & Cuevas, P. Facundo & Parisotto, Luca, 2021. "Children on the move: Progressive redistribution of humanitarian cash transfers among refugees," Journal of Development Economics, Elsevier, vol. 153(C).
    14. Han, Kevin & Basse, Guillaume & Bojinov, Iavor, 2024. "Population interference in panel experiments," Journal of Econometrics, Elsevier, vol. 238(1).
    15. Guido W. Imbens, 2022. "Causality in Econometrics: Choice vs Chance," Econometrica, Econometric Society, vol. 90(6), pages 2541-2566, November.
    16. Andrea Mercatanti & Fan Li, 2017. "Do debit cards decrease cash demand?: causal inference and sensitivity analysis using principal stratification," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 66(4), pages 759-776, August.
    17. Sun, Shanxia & Delgado, Michael & Khanna, Neha, 2017. "Hybrid Vehicles and Household Driving Behavior: Implications for Miles Traveled and Gasoline Consumption," 2017 Annual Meeting, July 30-August 1, Chicago, Illinois 258502, Agricultural and Applied Economics Association.
    18. Guido W. Imbens, 2020. "Potential Outcome and Directed Acyclic Graph Approaches to Causality: Relevance for Empirical Practice in Economics," Journal of Economic Literature, American Economic Association, vol. 58(4), pages 1129-1179, December.
    19. Peter Z. Schochet, 2018. "Design-Based Estimators for Average Treatment Effects for Multi-Armed RCTs," Journal of Educational and Behavioral Statistics, , vol. 43(5), pages 568-593, October.
    20. Lu, Jiannan, 2016. "On randomization-based and regression-based inferences for 2K factorial designs," Statistics & Probability Letters, Elsevier, vol. 112(C), pages 72-78.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:istatr:v:88:y:2020:i:1:p:101-121. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://edirc.repec.org/data/isiiinl.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.