IDEAS home Printed from https://ideas.repec.org/a/spr/aphecp/v14y2016i4d10.1007_s40258-016-0249-y.html
   My bibliography  Save this article

Difference-in-Differences Method in Comparative Effectiveness Research: Utility with Unbalanced Groups

Author

Listed:
  • Huanxue Zhou

    (KMK Consulting, Inc.)

  • Christopher Taber

    (University of Wisconsin)

  • Steve Arcona

    (Novartis Pharmaceuticals Corporation)

  • Yunfeng Li

    (Novartis Pharmaceuticals Corporation)

Abstract

Background Comparative effectiveness research (CER) often includes observational studies utilizing administrative data. Multiple conditioning methods can be used for CER to adjust for group differences, including difference-in-differences (DiD) estimation. Objective This study presents DiD and demonstrates how to apply this conditioning method to estimate treatment outcomes in the CER setting by utilizing the MarketScan® Databases for multiple sclerosis (MS) patients receiving different therapies. Methods The sample included 6762 patients, with 363 in the Test Cohort [glatiramer acetate (GA) switched to fingolimod (FTY)] and 6399 in the Control Cohort (GA only, no switch) from a US administrative claims database. A trend analysis was conducted to rule out concerns regarding regression to the mean and to compare relapse rates among treatment cohorts. DiD analysis was used to enable comparisons among the Test and Control Cohorts. Logistic regression was used to estimate the probability of relapse after switching from GA to FTY, and to compare group differences in the pre- and post-index periods. Results Crude DiD analysis showed that in the pre-index period more patients in the Test Cohort experienced an MS relapse and had a higher mean number of relapses than in the Control Cohort. During the pre-index period, numeric and relative data for MS relapses in patients in the Test Cohort were significantly higher than in the Control Cohort, while no significant between-group differences emerged during the post-index period. Generalized linear modeling with DiD regression estimation showed that the mean number of MS relapses decreased significantly in the post-index period among patients in the Test Cohort compared with patients in the Control Cohort. Conclusion In this study, an MS population was utilized to demonstrate how DiD can be applied to estimate treatment effects in a heterogeneous population, where the Test and Control Cohorts varied greatly. The results show that DiD offers a robust method for comparing diverse cohorts when other risk-adjustment methods may not be adequate.

Suggested Citation

  • Huanxue Zhou & Christopher Taber & Steve Arcona & Yunfeng Li, 2016. "Difference-in-Differences Method in Comparative Effectiveness Research: Utility with Unbalanced Groups," Applied Health Economics and Health Policy, Springer, vol. 14(4), pages 419-429, August.
    • Handle: RePEc:spr:aphecp:v:14:y:2016:i:4:d:10.1007_s40258-016-0249-y
    DOI: 10.1007/s40258-016-0249-y
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s40258-016-0249-y
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s40258-016-0249-y?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Ashenfelter, Orley C, 1978. "Estimating the Effect of Training Programs on Earnings," The Review of Economics and Statistics, MIT Press, vol. 60(1), pages 47-57, February.
    2. James J. Heckman & Jeffrey A. Smith, 1999. "The Pre-Program Earnings Dip and the Determinants of Participation in a Social Program: Implications for Simple Program Evaluation Strategies," NBER Working Papers 6983, National Bureau of Economic Research, Inc.
    3. Heckman, James J & Smith, Jeffrey A, 1999. "The Pre-programme Earnings Dip and the Determinants of Participation in a Social Programme. Implications for Simple Programme Evaluation Strategies," Economic Journal, Royal Economic Society, vol. 109(457), pages 313-348, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Ambarish Dutta & Sarthak Pattanaik & Rajendra Choudhury & Pritish Nanda & Suvanand Sahu & Rajendra Panigrahi & Bijaya K Padhi & Krushna Chandra Sahoo & P R Mishra & Pinaki Panigrahi & Daisy Lekharu & , 2018. "Impact of involvement of non-formal health providers on TB case notification among migrant slum-dwelling populations in Odisha, India," PLOS ONE, Public Library of Science, vol. 13(5), pages 1-16, May.
    2. Li Hao, 2022. "Impact of Relaxing the Hukou Constraints on Return Migration Intentions: Evidence from China," Population Research and Policy Review, Springer;Southern Demographic Association (SDA), vol. 41(2), pages 583-607, April.
    3. Miraldo, Marisa & Propper, Carol & Williams, Rachael I., 2018. "The impact of publicly subsidised health insurance on access, behavioural risk factors and disease management," Social Science & Medicine, Elsevier, vol. 217(C), pages 135-151.
    4. Song, Mingjie & Wu, Yuling & Chen, Lu, 2020. "Does the land titling program promote rural housing land transfer in China? Evidence from household surveys in Hubei Province," Land Use Policy, Elsevier, vol. 97(C).
    5. Pécastaing, Nicolas & Chávez, Carlos, 2020. "The impact of El Niño phenomenon on dry forest-dependent communities' welfare in the northern coast of Peru," Ecological Economics, Elsevier, vol. 178(C).
    6. Vy Thi Nhat Nguyen & Takashi Zaitsu & Akiko Oshiro & Tai Tan Tran & Yen Hoang Thi Nguyen & Yoko Kawaguchi & Jun Aida, 2021. "Impact of School-Based Oral Health Education on Vietnamese Adolescents: A 6-Month Study," IJERPH, MDPI, vol. 18(5), pages 1-12, March.
    7. Hubert Kryszk & Krystyna Kurowska & Renata Marks-Bielska, 2024. "The Importance of Measures Undertaken to Improve the Quality of Life in the Problem Areas: A Case Study in Warmia and Mazury Region in Poland," Sustainability, MDPI, vol. 16(16), pages 1-19, August.
    8. Jessica Yu & Paul Gustafson & Martino Tran & Michael Brauer, 2022. "Assessing Trade-Offs and Optimal Ranges of Density for Life Expectancy and 12 Causes of Mortality in Metro Vancouver, Canada, 1990–2016," IJERPH, MDPI, vol. 19(5), pages 1-17, March.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Gabrielle Wills, 2016. "Principal leadership changes in South Africa: Investigating their consequences for school performance," Working Papers 01/2016, Stellenbosch University, Department of Economics.
    2. Stenberg, Anders & Westerlund, Olle, 2016. "Flexibility at a cost – Should governments stimulate tertiary education for adults?," The Journal of the Economics of Ageing, Elsevier, vol. 7(C), pages 69-86.
    3. Regner, Hakan, 2002. "A nonexperimental evaluation of training programs for the unemployed in Sweden," Labour Economics, Elsevier, vol. 9(2), pages 187-206, April.
    4. Lechner Michael & Miquel Ruth & Wunsch Conny, 2007. "The Curse and Blessing of Training the Unemployed in a Changing Economy: The Case of East Germany After Unification," German Economic Review, De Gruyter, vol. 8(4), pages 468-509, December.
    5. Raphael W. Bostic & Allen C. Prohofsky, 2006. "Enterprise Zones and Individual Welfare: A Case Study of California," Journal of Regional Science, Wiley Blackwell, vol. 46(2), pages 175-203, May.
    6. Martins, Pedro S., 2017. "Can Non-Cognitive Skills Programs Improve Achievement? Quasi-Experimental Evidence from EPIS," GLO Discussion Paper Series 105, Global Labor Organization (GLO).
    7. Knut Løyland & Oddbjørn Raaum & Gaute Torsvik & Arnstein Øvrum, 2024. "Evaluating compliance gains of expanding tax enforcement," Economica, London School of Economics and Political Science, vol. 91(361), pages 142-162, January.
    8. Richard Blundell & Monica Costa Dias, 2009. "Alternative Approaches to Evaluation in Empirical Microeconomics," Journal of Human Resources, University of Wisconsin Press, vol. 44(3).
    9. Gueorgui Kambourov & Iourii Manovskii & Miana Plesca, 2020. "Occupational mobility and the returns to training," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 53(1), pages 174-211, February.
    10. Fredrik Andersson & Harry J. Holzer & Julia I. Lane & David Rosenblum & Jeffrey Smith, 2024. "Does Federally Funded Job Training Work? Nonexperimental Estimates of WIA Training Impacts Using Longitudinal Data on Workers and Firms," Journal of Human Resources, University of Wisconsin Press, vol. 59(4), pages 1244-1283.
    11. Chen, Yiu Por (Vincent), 2009. "Cream-Skimmer or Underdog? Labor Type Selectivity, Pre-Program Wage, and Rural Labor Training Program Outcome," IZA Discussion Papers 3979, Institute of Labor Economics (IZA).
    12. Christine Ryan & John Wilson & William Fulton, 2003. "The Impact of Urban Growth Boundaries on Future Urbanization," Working Paper 8610, USC Lusk Center for Real Estate.
    13. Alonso, José M. & Andrews, Rhys & Jorda, Vanesa, 2019. "Do neighbourhood renewal programs reduce crime rates? Evidence from England," Journal of Urban Economics, Elsevier, vol. 110(C), pages 51-69.
    14. Steven Raphael & Michael A. Stoll, 2006. "Evaluating the Effectiveness of the Massachusetts Workforce Development System Using No-Shows as a Nonexperimental Comparison Group," Evaluation Review, , vol. 30(4), pages 379-429, August.
    15. Jochen Kluve & Hartmut Lehmann & Christoph M. Schmidt, 2000. "Disentangling Treatment Effects of Polish Active Labour Market Policies: Evidence from Matched Samples," CERT Discussion Papers 0007, Centre for Economic Reform and Transformation, Heriot Watt University.
    16. Mario Liebensteiner, 2014. "Estimating the Income Gain of Seasonal Labor Migration," Review of Development Economics, Wiley Blackwell, vol. 18(4), pages 667-680, November.
    17. Hagen, Tobias, 2016. "Econometric Evaluation of a Placement Coaching Program for Recipients of Disability Insurance Benefits in Switzerland," VfS Annual Conference 2016 (Augsburg): Demographic Change 145736, Verein für Socialpolitik / German Economic Association.
    18. Robert S. Chirinko & Daniel J. Wilson, 2023. "Fiscal Foresight and Perverse Distortions to Firm Behavior: Anticipatory Dips and Compensating Rebounds," CESifo Working Paper Series 10373, CESifo.
    19. Robert S. Chirinko & Daniel J. Wilson, 2023. "Job Creation Tax Credits, Fiscal Foresight, and Job Growth: Evidence from US States," National Tax Journal, University of Chicago Press, vol. 76(3), pages 481-523.
    20. Celeste K. Carruthers & Thomas Sanford, 2015. "Way Station or Launching Pad? Unpacking the Returns to Postsecondary Adult Education," Working Papers 2015-02, University of Tennessee, Department of Economics.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:aphecp:v:14:y:2016:i:4:d:10.1007_s40258-016-0249-y. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.