IDEAS home Printed from https://ideas.repec.org/a/taf/jnlasa/v110y2015i509p205-217.html
   My bibliography  Save this article

Bahadur Efficiency of Sensitivity Analyses in Observational Studies

Author

Listed:
  • Paul R. Rosenbaum

Abstract

An observational study draws inferences about treatment effects when treatments are not randomly assigned, as they would be in a randomized experiment. The naive analysis of an observational study assumes that adjustments for measured covariates suffice to remove bias from nonrandom treatment assignment. A sensitivity analysis in an observational study determines the magnitude of bias from nonrandom treatment assignment that would need to be present to alter the qualitative conclusions of the naive analysis, say leading to the acceptance of a null hypothesis rejected in the naive analysis. Observational studies vary greatly in their sensitivity to unmeasured biases, but a poor choice of test statistic can lead to an exaggerated report of sensitivity to bias. The Bahadur efficiency of a sensitivity analysis is introduced, calculated, and connected to established concepts, such as the power of a sensitivity analysis and the design sensitivity. The Bahadur slope equals zero when the sensitivity parameter equals the design sensitivity, but the Bahadur slope permits more refined distinctions. Specifically, the Bahadur relative efficiency can also compare the relative performance of two test statistics at a value of the sensitivity parameter below the minimum of their design sensitivities. Adaptive procedures that combine several tests can achieve the best design sensitivity and the best Bahadur slope of their component tests. Ultimately, in sufficiently large sample sizes, design sensitivity is more important than efficiency for the power of a sensitivity analysis, and the exponential rate at which rate design sensitivity overtakes efficiency is characterized.

Suggested Citation

  • Paul R. Rosenbaum, 2015. "Bahadur Efficiency of Sensitivity Analyses in Observational Studies," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 110(509), pages 205-217, March.
    • Handle: RePEc:taf:jnlasa:v:110:y:2015:i:509:p:205-217
    DOI: 10.1080/01621459.2014.960968
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1080/01621459.2014.960968
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1080/01621459.2014.960968?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Jesse Y. Hsu & Dylan S. Small, 2013. "Calibrating Sensitivity Analyses to Observed Covariates in Observational Studies," Biometrics, The International Biometric Society, vol. 69(4), pages 803-811, December.
    2. Guildo W. Imbens, 2003. "Sensitivity to Exogeneity Assumptions in Program Evaluation," American Economic Review, American Economic Association, vol. 93(2), pages 126-132, May.
    3. Rosenbaum, Paul R., 2010. "Design Sensitivity and Efficiency in Observational Studies," Journal of the American Statistical Association, American Statistical Association, vol. 105(490), pages 692-702.
    4. Small, Dylan S., 2007. "Sensitivity Analysis for Instrumental Variables Regression With Overidentifying Restrictions," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 1049-1058, September.
    5. Dylan S. Small & Jing Cheng & M. Elizabeth Halloran & Paul R. Rosenbaum, 2013. "Case Definition and Design Sensitivity," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 108(504), pages 1457-1468, December.
    6. Paul R. Rosenbaum, 2007. "Sensitivity Analysis for m-Estimates, Tests, and Confidence Intervals in Matched Observational Studies," Biometrics, The International Biometric Society, vol. 63(2), pages 456-464, June.
    7. Peter B. Gilbert & Ronald J. Bosch & Michael G. Hudgens, 2003. "Sensitivity Analysis for the Assessment of Causal Vaccine Effects on Viral Load in HIV Vaccine Trials," Biometrics, The International Biometric Society, vol. 59(3), pages 531-541, September.
    8. Paul R. Rosenbaum, 2004. "Design sensitivity in observational studies," Biometrika, Biometrika Trust, vol. 91(1), pages 153-164, March.
    9. Rosenbaum, Paul R. & Silber, Jeffrey H., 2009. "Amplification of Sensitivity Analysis in Matched Observational Studies," Journal of the American Statistical Association, American Statistical Association, vol. 104(488), pages 1398-1405.
    10. Jesse Y. Hsu & Dylan S. Small & Paul R. Rosenbaum, 2013. "Effect Modification and Design Sensitivity in Observational Studies," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 108(501), pages 135-148, March.
    11. P. R. Rosenbaum, 2012. "Testing one hypothesis twice in observational studies," Biometrika, Biometrika Trust, vol. 99(4), pages 763-774.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Siyu Heng & Dylan S. Small & Paul R. Rosenbaum, 2020. "Finding the strength in a weak instrument in a study of cognitive outcomes produced by Catholic high schools," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 183(3), pages 935-958, June.
    2. Paul R. Rosenbaum, 2015. "Some Counterclaims Undermine Themselves in Observational Studies," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 110(512), pages 1389-1398, December.
    3. Xueping Li & Xingmin Shi & Yuhan Qin, 2024. "Exploring the Effects of Farmland Transfer on Farm Household Well-Being: Evidence from Ore–Agriculture Compound Areas in Northwest China," Land, MDPI, vol. 13(12), pages 1-23, November.
    4. Kwonsang Lee & Dylan S. Small & Paul R. Rosenbaum, 2018. "A powerful approach to the study of moderate effect modification in observational studies," Biometrics, The International Biometric Society, vol. 74(4), pages 1161-1170, December.
    5. Ruoqi Yu, 2021. "Evaluating and improving a matched comparison of antidepressants and bone density," Biometrics, The International Biometric Society, vol. 77(4), pages 1276-1288, December.
    6. Paul R. Rosenbaum, 2023. "A second evidence factor for a second control group," Biometrics, The International Biometric Society, vol. 79(4), pages 3968-3980, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kwonsang Lee & Dylan S. Small & Paul R. Rosenbaum, 2018. "A powerful approach to the study of moderate effect modification in observational studies," Biometrics, The International Biometric Society, vol. 74(4), pages 1161-1170, December.
    2. Siyu Heng & Hyunseung Kang & Dylan S. Small & Colin B. Fogarty, 2021. "Increasing power for observational studies of aberrant response: An adaptive approach," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 83(3), pages 482-504, July.
    3. Paul R. Rosenbaum, 2013. "Impact of Multiple Matched Controls on Design Sensitivity in Observational Studies," Biometrics, The International Biometric Society, vol. 69(1), pages 118-127, March.
    4. Paul R. Rosenbaum, 2011. "A New u-Statistic with Superior Design Sensitivity in Matched Observational Studies," Biometrics, The International Biometric Society, vol. 67(3), pages 1017-1027, September.
    5. Paul R. Rosenbaum, 2015. "Some Counterclaims Undermine Themselves in Observational Studies," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 110(512), pages 1389-1398, December.
    6. Bo Zhang & Eric J. Tchetgen Tchetgen, 2022. "A semi‐parametric approach to model‐based sensitivity analysis in observational studies," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 185(S2), pages 668-691, December.
    7. Siyu Heng & Dylan S. Small & Paul R. Rosenbaum, 2020. "Finding the strength in a weak instrument in a study of cognitive outcomes produced by Catholic high schools," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 183(3), pages 935-958, June.
    8. Xuran Wang & Yang Jiang & Nancy R. Zhang & Dylan S. Small, 2018. "Sensitivity analysis and power for instrumental variable studies," Biometrics, The International Biometric Society, vol. 74(4), pages 1150-1160, December.
    9. Ben B. Hansen & Paul R. Rosenbaum & Dylan S. Small, 2014. "Clustered Treatment Assignments and Sensitivity to Unmeasured Biases in Observational Studies," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 109(505), pages 133-144, March.
    10. Raiden B. Hasegawa & Sameer K. Deshpande & Dylan S. Small & Paul R. Rosenbaum, 2020. "Causal Inference With Two Versions of Treatment," Journal of Educational and Behavioral Statistics, , vol. 45(4), pages 426-445, August.
    11. Haensch, Anna-Carolina & Drechsler, Jörg & Bernhard, Sarah, 2020. "TippingSens: An R Shiny Application to Facilitate Sensitivity Analysis for Causal Inference Under Confounding," IAB-Discussion Paper 202029, Institut für Arbeitsmarkt- und Berufsforschung (IAB), Nürnberg [Institute for Employment Research, Nuremberg, Germany].
    12. Paul R. Rosenbaum & Dylan S. Small, 2017. "An adaptive Mantel–Haenszel test for sensitivity analysis in observational studies," Biometrics, The International Biometric Society, vol. 73(2), pages 422-430, June.
    13. Bo Zhang & Dylan S. Small, 2020. "A calibrated sensitivity analysis for matched observational studies with application to the effect of second‐hand smoke exposure on blood lead levels in children," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 69(5), pages 1285-1305, November.
    14. Samuel D. Pimentel & Dylan S. Small & Paul R. Rosenbaum, 2016. "Constructed Second Control Groups and Attenuation of Unmeasured Biases," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 111(515), pages 1157-1167, July.
    15. Paul R. Rosenbaum, 2023. "A second evidence factor for a second control group," Biometrics, The International Biometric Society, vol. 79(4), pages 3968-3980, December.
    16. Richard A. Ashley & Guo Li, 2013. "Re-Examining the Impact of Housing Wealth and Stock Wealth on Household Spending: Does Persistence in Wealth Changes Matter?," Working Papers e07-39, Virginia Polytechnic Institute and State University, Department of Economics.
    17. Doko Tchatoka, Firmin, 2012. "Specification tests with weak and invalid instruments," Working Papers 15063, University of Tasmania, Tasmanian School of Business and Economics, revised 26 Jun 2012.
    18. Minna Genbäck & Elena Stanghellini & Xavier Luna, 2015. "Uncertainty intervals for regression parameters with non-ignorable missingness in the outcome," Statistical Papers, Springer, vol. 56(3), pages 829-847, August.
    19. Giovanni Nattino & Bo Lu, 2018. "Model assisted sensitivity analyses for hidden bias with binary outcomes," Biometrics, The International Biometric Society, vol. 74(4), pages 1141-1149, December.
    20. Xinkun Nie & Guido Imbens & Stefan Wager, 2021. "Covariate Balancing Sensitivity Analysis for Extrapolating Randomized Trials across Locations," Papers 2112.04723, arXiv.org.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:taf:jnlasa:v:110:y:2015:i:509:p:205-217. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Longhurst (email available below). General contact details of provider: http://www.tandfonline.com/UASA20 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.