IDEAS home Printed from https://ideas.repec.org/a/eee/socmed/v258y2020ics0277953620302409.html
   My bibliography  Save this article

Measuring the impact of national guidelines: What methods can be used to uncover time-varying effects for healthcare evaluations?

Author

Listed:
  • Price, Sarah
  • Zhang, Xiaohui
  • Spencer, Anne

Abstract

We examine the suitability of three methods using patient-level data to evaluate the time-varying impacts of national healthcare guidelines. Such guidelines often codify progressive change and are implemented gradually; for example, National Institute for Health and Care Excellence (NICE) suspected-cancer referral guidelines. These were revised on June 23, 2015, to include more cancer symptoms and test results (“features”), partly reflecting changing practice. We explore the time-varying impact of guideline revision on time to colorectal cancer diagnosis, which is linked to improved outcomes in decision-analytic models. We included 11,842 patients diagnosed in 01/01/2006–31/12/2017 in the Clinical Practice Research Datalink with England cancer registry data linkage. Patients were classified by whether their first pre-diagnostic cancer feature was in the original guidelines (NICE-2005) or was added during the revision (NICE-2015-only). Outcome was diagnostic interval: time from first cancer feature to diagnosis. All analyses adjusted for age and sex. Two difference-in-differences analyses used either a Pre (01/08/2012–31/12/2014, n = 2243) and Post (01/08/2015–31/12/2017, n = 1017) design, or event-study cohorts (2006–2017 vs 2015) to estimate change in diagnostic interval attributable to official implementation of the revised guidelines. A semiparametric varying-coefficient model analysed the difference in diagnostic interval between the NICE groups over time. After model estimation, primary and broader treatment effects of guideline content and implementation were measured. The event-study difference-in-differences and the semiparametric varying-coefficient methods showed that shorter diagnostic intervals were attributable to official implementation of the revised guidelines. This impact was only detectable by pre-to-post difference-in-differences when the pre/post periods were selected according to the estimation results from the varying-coefficient model. Formal tests of the parametric models, which are special cases of the semiparametric model, suggest that they are misspecified. We conclude that the semiparametric method is well suited to explore the time-varying impacts of guidelines codifying progressive change.

Suggested Citation

  • Price, Sarah & Zhang, Xiaohui & Spencer, Anne, 2020. "Measuring the impact of national guidelines: What methods can be used to uncover time-varying effects for healthcare evaluations?," Social Science & Medicine, Elsevier, vol. 258(C).
    • Handle: RePEc:eee:socmed:v:258:y:2020:i:c:s0277953620302409
    DOI: 10.1016/j.socscimed.2020.113021
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0277953620302409
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.socscimed.2020.113021?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Card, David & Krueger, Alan B, 1994. "Minimum Wages and Employment: A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania," American Economic Review, American Economic Association, vol. 84(4), pages 772-793, September.
    2. Zarek C. Brot-Goldberg & Amitabh Chandra & Benjamin R. Handel & Jonathan T. Kolstad, 2017. "What does a Deductible Do? The Impact of Cost-Sharing on Health Care Prices, Quantities, and Spending Dynamics," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 132(3), pages 1261-1318.
    3. Marianne P. Bitler & Christopher S. Carpenter, 2016. "Health Insurance Mandates, Mammography, and Breast Cancer Diagnoses," American Economic Journal: Economic Policy, American Economic Association, vol. 8(3), pages 39-68, August.
    4. Kolstad, Jonathan T. & Kowalski, Amanda E., 2012. "The impact of health care reform on hospital and preventive care: Evidence from Massachusetts," Journal of Public Economics, Elsevier, vol. 96(11), pages 909-929.
    5. Feng, Guohua & Gao, Jiti & Peng, Bin & Zhang, Xiaohui, 2017. "A varying-coefficient panel data model with fixed effects: Theory and an application to US commercial banks," Journal of Econometrics, Elsevier, vol. 196(1), pages 68-82.
    6. Noémi Kreif & Richard Grieve & Dominik Hangartner & Alex James Turner & Silviya Nikolova & Matt Sutton, 2016. "Examination of the Synthetic Control Method for Evaluating Health Policies with Multiple Treated Units," Health Economics, John Wiley & Sons, Ltd., vol. 25(12), pages 1514-1528, December.
    7. Hayfield, Tristen & Racine, Jeffrey S., 2008. "Nonparametric Econometrics: The np Package," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 27(i05).
    8. Alpert, Abby, 2016. "The anticipatory effects of Medicare Part D on drug utilization," Journal of Health Economics, Elsevier, vol. 49(C), pages 28-45.
    9. Juan M. Rodriguez‐Poo & Alexandra Soberon, 2014. "Direct semi‐parametric estimation of fixed effects panel data varying coefficient models," Econometrics Journal, Royal Economic Society, vol. 17(1), pages 107-138, February.
    10. Li, Qi, et al, 2002. "Semiparametric Smooth Coefficient Models," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(3), pages 412-422, July.
    11. Li, Qi & Racine, Jeffrey S., 2010. "Smooth Varying-Coefficient Estimation And Inference For Qualitative And Quantitative Data," Econometric Theory, Cambridge University Press, vol. 26(6), pages 1607-1637, December.
    12. Alberto Abadie, 2005. "Semiparametric Difference-in-Differences Estimators," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 72(1), pages 1-19.
    13. Sarah J. Reber, 2005. "Court Ordered Desegregation: Successes and Failures Integrating American Schools since Brown versus Board of Education," Journal of Human Resources, University of Wisconsin Press, vol. 40(3).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Sun, Kai & Kumbhakar, Subal C. & Tveterås, Ragnar, 2015. "Productivity and efficiency estimation: A semiparametric stochastic cost frontier approach," European Journal of Operational Research, Elsevier, vol. 245(1), pages 194-202.
    2. Feng, Guohua & Gao, Jiti & Peng, Bin & Zhang, Xiaohui, 2017. "A varying-coefficient panel data model with fixed effects: Theory and an application to US commercial banks," Journal of Econometrics, Elsevier, vol. 196(1), pages 68-82.
    3. Diego A. Restrepo-Tobón & Subal C. Kumbhakar & Kai Sun, 2013. "Are U.S. Commercial Banks Too Big?," Documentos de Trabajo de Valor Público 10943, Universidad EAFIT.
    4. Subal Kumbhakar & Kai Sun, 2012. "Estimation of TFP growth: a semiparametric smooth coefficient approach," Empirical Economics, Springer, vol. 43(1), pages 1-24, August.
    5. Myoung-jae Lee & Yasuyuki Sawada, 2020. "Review on Difference in Differences," Korean Economic Review, Korean Economic Association, vol. 36, pages 135-173.
    6. Benjamin David, 2014. "Contribution of ICT on Labor Market Polarization: an Evolutionary Approach," EconomiX Working Papers 2014-25, University of Paris Nanterre, EconomiX.
    7. Thomas P. Triebs & Subal C. Kumbhakar, 2018. "Management in production: from unobserved to observed," Journal of Productivity Analysis, Springer, vol. 49(2), pages 111-121, June.
    8. Diego Restrepo-Tobón & Subal Kumbhakar & Kai Sun, 2015. "Obelix vs. Asterix: Size of US commercial banks and its regulatory challenge," Journal of Regulatory Economics, Springer, vol. 48(2), pages 125-168, October.
    9. Ting Ye & Ashkan Ertefaie & James Flory & Sean Hennessy & Dylan S. Small, 2023. "Instrumented difference‐in‐differences," Biometrics, The International Biometric Society, vol. 79(2), pages 569-581, June.
    10. Zhang, Wenyang & Li, Degui & Xia, Yingcun, 2015. "Estimation in generalised varying-coefficient models with unspecified link functions," Journal of Econometrics, Elsevier, vol. 187(1), pages 238-255.
    11. Nicholas Illenberger & Dylan S. Small & Pamela A. Shaw, 2019. "Regression to the Mean's Impact on the Synthetic Control Method: Bias and Sensitivity Analysis," Papers 1909.04706, arXiv.org.
    12. Eli Ben‐Michael & Avi Feller & Jesse Rothstein, 2022. "Synthetic controls with staggered adoption," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 84(2), pages 351-381, April.
    13. Das, Sonali & Racine, Jeffrey S., 2018. "Interactive nonparametric analysis of nonlinear systems," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 510(C), pages 290-301.
    14. Masserini, Lucio & Bini, Matilde & Lorenzoni, Valentina, 2024. "The effect of pricing policies on students’ use of university canteens," Socio-Economic Planning Sciences, Elsevier, vol. 94(C).
    15. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    16. Yang Ning & Sida Peng & Jing Tao, 2020. "Doubly Robust Semiparametric Difference-in-Differences Estimators with High-Dimensional Data," Papers 2009.03151, arXiv.org.
    17. Dmitry Arkhangelsky & Guido Imbens, 2023. "Causal Models for Longitudinal and Panel Data: A Survey," Papers 2311.15458, arXiv.org, revised Jun 2024.
    18. Olli Ropponen, 2014. "A note on the robustness of Card and Krueger (1994) and Neumark and Wascher (2000)," Empirical Economics, Springer, vol. 46(1), pages 307-316, February.
    19. Roth, Jonathan & Sant’Anna, Pedro H.C. & Bilinski, Alyssa & Poe, John, 2023. "What’s trending in difference-in-differences? A synthesis of the recent econometrics literature," Journal of Econometrics, Elsevier, vol. 235(2), pages 2218-2244.
    20. Valentina Hartarska & Christopher F. Parmeter & Denis Nadolnyak & Beibei Zhu, 2010. "Economies Of Scope For Microfinance: Differences Across Output Measures," Pacific Economic Review, Wiley Blackwell, vol. 15(4), pages 464-481, October.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:socmed:v:258:y:2020:i:c:s0277953620302409. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/315/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.