IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2304.11735.html
   My bibliography  Save this paper

Policy Learning under Biased Sample Selection

Author

Listed:
  • Lihua Lei
  • Roshni Sahoo
  • Stefan Wager

Abstract

Practitioners often use data from a randomized controlled trial to learn a treatment assignment policy that can be deployed on a target population. A recurring concern in doing so is that, even if the randomized trial was well-executed (i.e., internal validity holds), the study participants may not represent a random sample of the target population (i.e., external validity fails)--and this may lead to policies that perform suboptimally on the target population. We consider a model where observable attributes can impact sample selection probabilities arbitrarily but the effect of unobservable attributes is bounded by a constant, and we aim to learn policies with the best possible performance guarantees that hold under any sampling bias of this type. In particular, we derive the partial identification result for the worst-case welfare in the presence of sampling bias and show that the optimal max-min, max-min gain, and minimax regret policies depend on both the conditional average treatment effect (CATE) and the conditional value-at-risk (CVaR) of potential outcomes given covariates. To avoid finite-sample inefficiencies of plug-in estimates, we further provide an end-to-end procedure for learning the optimal max-min and max-min gain policies that does not require the separate estimation of nuisance parameters.

Suggested Citation

  • Lihua Lei & Roshni Sahoo & Stefan Wager, 2023. "Policy Learning under Biased Sample Selection," Papers 2304.11735, arXiv.org.
  • Handle: RePEc:arx:papers:2304.11735
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2304.11735
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Toru Kitagawa & Aleksey Tetenov, 2018. "Who Should Be Treated? Empirical Welfare Maximization Methods for Treatment Choice," Econometrica, Econometric Society, vol. 86(2), pages 591-616, March.
    2. Susan Athey & Stefan Wager, 2021. "Policy Learning With Observational Data," Econometrica, Econometric Society, vol. 89(1), pages 133-161, January.
    3. repec:dau:papers:123456789/2278 is not listed on IDEAS
    4. Alfred Galichon & Ivar Ekeland & Marc Henry, 2009. "Comonotonic measures of multivariates risks," Working Papers hal-00401828, HAL.
    5. Charles F. Manski, 2011. "Choosing Treatment Policies Under Ambiguity," Annual Review of Economics, Annual Reviews, vol. 3(1), pages 25-49, September.
    6. Alfred Galichon & Ivar Ekeland & Marc Henry, 2009. "Comonotonic measures of multivariates risks," Working Papers hal-00401828, HAL.
    7. repec:hal:wpspec:info:hdl:2441/5rkqqmvrn4tl22s9mc4b1h6b4 is not listed on IDEAS
    8. Charles F Manski, 2007. "Adaptive Minimax-Regret Treatment Choice, with Application to Drug Approval," Levine's Working Paper Archive 122247000000001404, David K. Levine.
    9. Lars Peter Hansen & Thomas J Sargent, 2014. "Robust Control and Model Uncertainty," World Scientific Book Chapters, in: UNCERTAINTY WITHIN ECONOMIC MODELS, chapter 5, pages 145-154, World Scientific Publishing Co. Pte. Ltd..
    10. Manski, Charles F., 2000. "Identification problems and decisions under ambiguity: Empirical analysis of treatment response and normative analysis of treatment choice," Journal of Econometrics, Elsevier, vol. 95(2), pages 415-442, April.
    11. Charles F. Manski, 2004. "Statistical Treatment Rules for Heterogeneous Populations," Econometrica, Econometric Society, vol. 72(4), pages 1221-1246, July.
    12. Manski, Charles F., 2007. "Minimax-regret treatment choice with missing outcome data," Journal of Econometrics, Elsevier, vol. 139(1), pages 105-115, July.
    13. Elizabeth A. Stuart & Stephen R. Cole & Catherine P. Bradshaw & Philip J. Leaf, 2011. "The use of propensity scores to assess the generalizability of results from randomized trials," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 174(2), pages 369-386, April.
    14. Peter M. Aronow & Donald K. K. Lee, 2013. "Interval estimation of population means under unknown but bounded probabilities of sample selection," Biometrika, Biometrika Trust, vol. 100(1), pages 235-240.
    15. Gerber, Alan S. & Green, Donald P. & Larimer, Christopher W., 2008. "Social Pressure and Voter Turnout: Evidence from a Large-Scale Field Experiment," American Political Science Review, Cambridge University Press, vol. 102(1), pages 33-48, February.
    16. Bhattacharya, Debopam & Dupas, Pascaline, 2012. "Inferring welfare maximizing treatment assignment under budget constraints," Journal of Econometrics, Elsevier, vol. 167(1), pages 168-196.
    17. Xinkun Nie & Guido Imbens & Stefan Wager, 2021. "Covariate Balancing Sensitivity Analysis for Extrapolating Randomized Trials across Locations," Papers 2112.04723, arXiv.org.
    18. Samuel Higbee, 2022. "Policy Learning with New Treatments," Papers 2210.04703, arXiv.org, revised Sep 2023.
    19. repec:hal:spmain:info:hdl:2441/5rkqqmvrn4tl22s9mc4b1h6b4 is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Zhaonan Qu & Yongchan Kwon, 2024. "Distributionally Robust Instrumental Variables Estimation," Papers 2410.15634, arXiv.org, revised Dec 2024.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Charles F. Manski, 2021. "Econometrics for Decision Making: Building Foundations Sketched by Haavelmo and Wald," Econometrica, Econometric Society, vol. 89(6), pages 2827-2853, November.
    2. Toru Kitagawa & Sokbae Lee & Chen Qiu, 2022. "Treatment Choice with Nonlinear Regret," Papers 2205.08586, arXiv.org, revised Oct 2024.
    3. Garbero, Alessandra & Sakos, Grayson & Cerulli, Giovanni, 2023. "Towards data-driven project design: Providing optimal treatment rules for development projects," Socio-Economic Planning Sciences, Elsevier, vol. 89(C).
    4. Timothy Christensen & Hyungsik Roger Moon & Frank Schorfheide, 2020. "Robust Forecasting," Papers 2011.03153, arXiv.org, revised Dec 2020.
    5. Yuehao Bai & Azeem M. Shaikh & Max Tabord-Meehan, 2024. "A Primer on the Analysis of Randomized Experiments and a Survey of some Recent Advances," Papers 2405.03910, arXiv.org.
    6. Chunrong Ai & Yue Fang & Haitian Xie, 2024. "Data-driven Policy Learning for Continuous Treatments," Papers 2402.02535, arXiv.org, revised Nov 2024.
    7. Manski, Charles F., 2023. "Probabilistic prediction for binary treatment choice: With focus on personalized medicine," Journal of Econometrics, Elsevier, vol. 234(2), pages 647-663.
    8. Kock, Anders Bredahl & Preinerstorfer, David & Veliyev, Bezirgen, 2023. "Treatment recommendation with distributional targets," Journal of Econometrics, Elsevier, vol. 234(2), pages 624-646.
    9. Shosei Sakaguchi, 2021. "Estimation of Optimal Dynamic Treatment Assignment Rules under Policy Constraints," Papers 2106.05031, arXiv.org, revised Aug 2024.
    10. Stefanie Behncke & Markus Frölich & Michael Lechner, 2009. "Targeting Labour Market Programmes - Results from a Randomized Experiment," Swiss Journal of Economics and Statistics (SJES), Swiss Society of Economics and Statistics (SSES), vol. 145(III), pages 221-268, September.
    11. Toru Kitagawa & Weining Wang & Mengshan Xu, 2022. "Policy Choice in Time Series by Empirical Welfare Maximization," Papers 2205.03970, arXiv.org, revised Dec 2024.
    12. Yu-Chang Chen & Haitian Xie, 2022. "Personalized Subsidy Rules," Papers 2202.13545, arXiv.org, revised Mar 2022.
    13. Johannes Haushofer & Paul Niehaus & Carlos Paramo & Edward Miguel & Michael W. Walker, 2022. "Targeting Impact versus Deprivation," NBER Working Papers 30138, National Bureau of Economic Research, Inc.
    14. Juliano Assunção & Robert McMillan & Joshua Murphy & Eduardo Souza-Rodrigues, 2019. "Optimal Environmental Targeting in the Amazon Rainforest," NBER Working Papers 25636, National Bureau of Economic Research, Inc.
    15. Yuya Sasaki & Takuya Ura, 2020. "Welfare Analysis via Marginal Treatment Effects," Papers 2012.07624, arXiv.org.
    16. Liyang Sun, 2021. "Empirical Welfare Maximization with Constraints," Papers 2103.15298, arXiv.org, revised Sep 2024.
    17. Timothy Christensen & Hyungsik Roger Moon & Frank Schorfheide, 2022. "Optimal Decision Rules when Payoffs are Partially Identified," Papers 2204.11748, arXiv.org, revised May 2023.
    18. Juan Carlos Perdomo, 2023. "The Relative Value of Prediction in Algorithmic Decision Making," Papers 2312.08511, arXiv.org, revised May 2024.
    19. Davide Viviano & Jess Rudder, 2020. "Policy design in experiments with unknown interference," Papers 2011.08174, arXiv.org, revised May 2024.
    20. Toru Kitagawa & Guanyi Wang, 2021. "Who should get vaccinated? Individualized allocation of vaccines over SIR network," CeMMAP working papers CWP28/21, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2304.11735. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.