IDEAS home Printed from https://ideas.repec.org/a/bla/biomet/v79y2023i1p475-487.html
   My bibliography  Save this article

Sensitivity analyses informed by tests for bias in observational studies

Author

Listed:
  • Paul R. Rosenbaum

Abstract

In an observational study, the treatment received and the outcome exhibited may be associated in the absence of an effect caused by the treatment, even after controlling for observed covariates. Two tactics are common: (i) a test for unmeasured bias may be obtained using a secondary outcome for which the effect is known and (ii) a sensitivity analysis may explore the magnitude of unmeasured bias that would need to be present to explain the observed association as something other than an effect caused by the treatment. Can such a test for unmeasured bias inform the sensitivity analysis? If the test for bias does not discover evidence of unmeasured bias, then ask: Are conclusions therefore insensitive to larger unmeasured biases? Conversely, if the test for bias does find evidence of bias, then ask: What does that imply about sensitivity to biases? This problem is formulated in a new way as a convex quadratically constrained quadratic program and solved on a large scale using interior point methods by a modern solver. That is, a convex quadratic function of N variables is minimized subject to constraints on linear and convex quadratic functions of these variables. The quadratic function that is minimized is a statistic for the primary outcome that is a function of the unknown treatment assignment probabilities. The quadratic function that constrains this minimization is a statistic for subsidiary outcome that is also a function of these same unknown treatment assignment probabilities. In effect, the first statistic is minimized over a confidence set for the unknown treatment assignment probabilities supplied by the unaffected outcome. This process avoids the mistake of interpreting the failure to reject a hypothesis as support for the truth of that hypothesis. The method is illustrated by a study of the effects of light daily alcohol consumption on high‐density lipoprotein (HDL) cholesterol levels. In this study, the method quickly optimizes a nonlinear function of N=800$N=800$ variables subject to linear and quadratic constraints. In the example, strong evidence of unmeasured bias is found using the subsidiary outcome, but, perhaps surprisingly, this finding makes the primary comparison insensitive to larger biases.

Suggested Citation

  • Paul R. Rosenbaum, 2023. "Sensitivity analyses informed by tests for bias in observational studies," Biometrics, The International Biometric Society, vol. 79(1), pages 475-487, March.
    • Handle: RePEc:bla:biomet:v:79:y:2023:i:1:p:475-487
    DOI: 10.1111/biom.13558
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/biom.13558
    Download Restriction: no
    File URL: https://libkey.io/10.1111/biom.13558?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Yurii Nesterov, 2018. "Lectures on Convex Optimization," Springer Optimization and Its Applications, Springer, edition 2, number 978-3-319-91578-4, July.
    2. West, S.G. & Duan, N. & Pequegnat, W. & Gaist, P. & Des Jarlais, D.C. & Holtgrave, D. & Szapocznik, J. & Fishbein, M. & Rapkin, B. & Clatts, M. & Mullen, P.D., 2008. "Alternatives to the randomized controlled trial," American Journal of Public Health, American Public Health Association, vol. 98(8), pages 1359-1366.
    3. B Karmakar & B French & D S Small, 2019. "Integrating the evidence from evidence factors in observational studies," Biometrika, Biometrika Trust, vol. 106(2), pages 353-367.
    4. Paul R. Rosenbaum, 2007. "Sensitivity Analysis for m-Estimates, Tests, and Confidence Intervals in Matched Observational Studies," Biometrics, The International Biometric Society, vol. 63(2), pages 456-464, June.
    5. Paul R. Rosenbaum, 2015. "Some Counterclaims Undermine Themselves in Observational Studies," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 110(512), pages 1389-1398, December.
    6. Colin B. Fogarty & Dylan S. Small, 2016. "Sensitivity Analysis for Multiple Comparisons in Matched Observational Studies Through Quadratically Constrained Linear Programming," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 111(516), pages 1820-1830, October.
    7. José R. Zubizarreta, 2012. "Using Mixed Integer Programming for Matching in an Observational Study of Kidney Failure After Surgery," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 107(500), pages 1360-1371, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Siyu Heng & Hyunseung Kang & Dylan S. Small & Colin B. Fogarty, 2021. "Increasing power for observational studies of aberrant response: An adaptive approach," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 83(3), pages 482-504, July.
    2. Paul R. Rosenbaum, 2023. "A second evidence factor for a second control group," Biometrics, The International Biometric Society, vol. 79(4), pages 3968-3980, December.
    3. Kwonsang Lee & Dylan S. Small & Paul R. Rosenbaum, 2018. "A powerful approach to the study of moderate effect modification in observational studies," Biometrics, The International Biometric Society, vol. 74(4), pages 1161-1170, December.
    4. Samuel D. Pimentel & Dylan S. Small & Paul R. Rosenbaum, 2016. "Constructed Second Control Groups and Attenuation of Unmeasured Biases," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 111(515), pages 1157-1167, July.
    5. Ruoqi Yu, 2021. "Evaluating and improving a matched comparison of antidepressants and bone density," Biometrics, The International Biometric Society, vol. 77(4), pages 1276-1288, December.
    6. Jason J. Sauppe & Sheldon H. Jacobson, 2017. "The role of covariate balance in observational studies," Naval Research Logistics (NRL), John Wiley & Sons, vol. 64(4), pages 323-344, June.
    7. Shota Takahashi & Mituhiro Fukuda & Mirai Tanaka, 2022. "New Bregman proximal type algorithms for solving DC optimization problems," Computational Optimization and Applications, Springer, vol. 83(3), pages 893-931, December.
    8. Guo, Chuanyin & Song, Qiwei & Yu, Ming-Miin & Zhang, Jian, 2024. "A digital economy development index based on an improved hierarchical data envelopment analysis approach," European Journal of Operational Research, Elsevier, vol. 316(3), pages 1146-1157.
    9. Andrés Elberg & Pedro M. Gardete & Rosario Macera & Carlos Noton, 2019. "Dynamic effects of price promotions: field evidence, consumer search, and supply-side implications," Quantitative Marketing and Economics (QME), Springer, vol. 17(1), pages 1-58, March.
    10. A. Scagliotti & P. Colli Franzone, 2022. "A piecewise conservative method for unconstrained convex optimization," Computational Optimization and Applications, Springer, vol. 81(1), pages 251-288, January.
    11. José R. Zubizarreta, 2015. "Stable Weights that Balance Covariates for Estimation With Incomplete Outcome Data," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 110(511), pages 910-922, September.
    12. Paul R Hernandez & Brittany Bloodhart & Rebecca T Barnes & Amanda S Adams & Sandra M Clinton & Ilana Pollack & Elaine Godfrey & Melissa Burt & Emily V Fischer, 2017. "Promoting professional identity, motivation, and persistence: Benefits of an informal mentoring program for female undergraduate students," PLOS ONE, Public Library of Science, vol. 12(11), pages 1-16, November.
    13. Armstrong, Christopher S. & Ittner, Christopher D. & Larcker, David F., 2010. "Corporate Governance, Compensation Consultants, and CEO Pay Levels," Research Papers 2068, Stanford University, Graduate School of Business.
    14. Xin Jiang & Lieven Vandenberghe, 2022. "Bregman primal–dual first-order method and application to sparse semidefinite programming," Computational Optimization and Applications, Springer, vol. 81(1), pages 127-159, January.
    15. Raiden B. Hasegawa & Sameer K. Deshpande & Dylan S. Small & Paul R. Rosenbaum, 2020. "Causal Inference With Two Versions of Treatment," Journal of Educational and Behavioral Statistics, , vol. 45(4), pages 426-445, August.
    16. Sahar Saeed & Erica E. M. Moodie & Erin C. Strumpf & Marina B. Klein, 2018. "Segmented generalized mixed effect models to evaluate health outcomes," International Journal of Public Health, Springer;Swiss School of Public Health (SSPH+), vol. 63(4), pages 547-551, May.
    17. Doikov, Nikita & Nesterov, Yurii, 2021. "Optimization Methods for Fully Composite Problems," LIDAM Discussion Papers CORE 2021001, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    18. Armstrong, Christopher S. & Blouin, Jennifer L. & Larcker, David F., 2012. "The incentives for tax planning," Journal of Accounting and Economics, Elsevier, vol. 53(1), pages 391-411.
    19. Díaz, Juan & Grau, Nicolás & Reyes, Tatiana & Rivera, Jorge, 2021. "The impact of grade retention on juvenile crime," Economics of Education Review, Elsevier, vol. 84(C).
    20. Md Saiful Islam & Md Sarowar Morshed & Gary J Young & Md Noor-E-Alam, 2019. "Robust policy evaluation from large-scale observational studies," PLOS ONE, Public Library of Science, vol. 14(10), pages 1-19, October.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:biomet:v:79:y:2023:i:1:p:475-487. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www.blackwellpublishing.com/journal.asp?ref=0006-341X .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.