IDEAS home Printed from https://ideas.repec.org/a/wly/hlthec/v21y2012i10p1234-1249.html
   My bibliography  Save this article

Measuring The Effect Of Policy Interventions At The Population Level: Some Methodological Concerns

Author

Listed:
  • Marco D. Huesch
  • Truls Østbye
  • Michael K. Ong

Abstract

Health policy evaluations estimate the response of population aggregate outcomes to interventions. However, clarity on the form of the expected causal relationship, the parameter identification strategy, and the mode of hypothesis testing is required to overcome a number of conceptual and methodological problems. We use the New Jersey statewide smoking ban as an example. We examine statewide admission rates for acute myocardial infarctions, strokes and lower limb fractures, and emergency room encounter rates for asthma exacerbations before and after the smoking ban. We discuss the identification options and show the sensitivity of estimates of the response function to different specifications of the stochastic and intervention components and to different modes of inference. Model misspecification is demonstrated by rolling Chow tests for structural breaks in repeated observations. Copyright © 2011 John Wiley & Sons, Ltd.

Suggested Citation

  • Marco D. Huesch & Truls Østbye & Michael K. Ong, 2012. "Measuring The Effect Of Policy Interventions At The Population Level: Some Methodological Concerns," Health Economics, John Wiley & Sons, Ltd., vol. 21(10), pages 1234-1249, October.
    • Handle: RePEc:wly:hlthec:v:21:y:2012:i:10:p:1234-1249
    DOI: 10.1002/hec.1783
    as

    Download full text from publisher

    File URL: https://doi.org/10.1002/hec.1783
    Download Restriction: no
    File URL: https://libkey.io/10.1002/hec.1783?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Kanaka D. Shetty & Thomas DeLeire & Chapin White & Jayanta Bhattacharya, 2011. "Changes in U.S. hospitalization and mortality rates following smoking bans," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 30(1), pages 6-28, December.
    2. Card, David & Krueger, Alan B, 1994. "Minimum Wages and Employment: A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania," American Economic Review, American Economic Association, vol. 84(4), pages 772-793, September.
    3. David Card, 1990. "The Impact of the Mariel Boatlift on the Miami Labor Market," ILR Review, Cornell University, ILR School, vol. 43(2), pages 245-257, January.
    4. Jones, A., 2007. "Applied Econometrics for Health Economists: A Practical Guide," Monographs, Office of Health Economics, number 000262.
    5. James J. Heckman & Edward Vytlacil, 2005. "Structural Equations, Treatment Effects, and Econometric Policy Evaluation," Econometrica, Econometric Society, vol. 73(3), pages 669-738, May.
    6. Ashenfelter, Orley C, 1978. "Estimating the Effect of Training Programs on Earnings," The Review of Economics and Statistics, MIT Press, vol. 60(1), pages 47-57, February.
    7. Stephen G. Donald & Kevin Lang, 2007. "Inference with Difference-in-Differences and Other Panel Data," The Review of Economics and Statistics, MIT Press, vol. 89(2), pages 221-233, May.
    8. Joshua D. Angrist & Jörn-Steffen Pischke, 2009. "Mostly Harmless Econometrics: An Empiricist's Companion," Economics Books, Princeton University Press, edition 1, number 8769.
    9. Pridemore, W.A. & Snowden, A.J., 2009. "Reduction in suicide mortality following a new national alcohol policy in Slovenia: An interrupted time-series analysis," American Journal of Public Health, American Public Health Association, vol. 99(5), pages 915-920.
    10. Ashenfelter, Orley & Card, David, 1985. "Using the Longitudinal Structure of Earnings to Estimate the Effect of Training Programs," The Review of Economics and Statistics, MIT Press, vol. 67(4), pages 648-660, November.
    11. Granger, C W J, 1969. "Investigating Causal Relations by Econometric Models and Cross-Spectral Methods," Econometrica, Econometric Society, vol. 37(3), pages 424-438, July.
    12. Marianne Bertrand & Esther Duflo & Sendhil Mullainathan, 2004. "How Much Should We Trust Differences-In-Differences Estimates?," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 119(1), pages 249-275.
    13. Sargent, R P & Shepard, R M & Glantz, Stanton A. Ph.D., 2004. "Reduced incidence of admissions for myocardial infarction associated with public smoking ban: before and after study," University of California at San Francisco, Center for Tobacco Control Research and Education qt3276d6r6, Center for Tobacco Control Research and Education, UC San Francisco.
    14. Wagstaff, Adam & Lindelow, Magnus & Jun, Gao & Ling, Xu & Juncheng, Qian, 2009. "Extending health insurance to the rural population: An impact evaluation of China's new cooperative medical scheme," Journal of Health Economics, Elsevier, vol. 28(1), pages 1-19, January.
    15. Adam Wagstaff, 2010. "Estimating health insurance impacts under unobserved heterogeneity: the case of Vietnam's health care fund for the poor," Health Economics, John Wiley & Sons, Ltd., vol. 19(2), pages 189-208, February.
    16. Ong, M K & Glantz, Stanton A. Ph.D., 2004. "Cardiovascular health and economic effects of smoke-free workplaces," University of California at San Francisco, Center for Tobacco Control Research and Education qt2ck7x753, Center for Tobacco Control Research and Education, UC San Francisco.
    17. Sung, H.-Y. & Hu, T.-W. & Ong, M. & Keeler, T.E. & Sheu, M.-L., 2005. "A major state tobacco tax increase, the master settlement agreement, and cigarette consumption: The California experience," American Journal of Public Health, American Public Health Association, vol. 95(6), pages 1030-1035.
    18. Gillings, D. & Makuc, D. & Siegel, E., 1981. "Analysis of interrupted time series mortality trends: An example to evaluate regionalized perinatal care," American Journal of Public Health, American Public Health Association, vol. 71(1), pages 38-46.
    19. Juster, H.R. & Loomis, B.R. & Hinman, T.M. & Farrelly, M.C. & Hyland, A. & Bauer, U.E. & Birkhead, G.S., 2007. "Declines in hospital admissions for acute myocardial infarction in new york state after implementation of a comprehensive smoking ban," American Journal of Public Health, American Public Health Association, vol. 97(11), pages 2035-2039.
    20. Anirban Basu & James J. Heckman & Salvador Navarro-Lozano & Sergio Urzua, 2007. "Use of instrumental variables in the presence of heterogeneity and self-selection: an application to treatments of breast cancer patients," Health Economics, John Wiley & Sons, Ltd., vol. 16(11), pages 1133-1157.
    21. Friedlander, Daniel & Robins, Philip K, 1995. "Evaluating Program Evaluations: New Evidence on Commonly Used Nonexperimental Methods," American Economic Review, American Economic Association, vol. 85(4), pages 923-937, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Rohan Sweeney & Duncan Mortimer, 2016. "Has the Swap Influenced Aid Flows in the Health Sector?," Health Economics, John Wiley & Sons, Ltd., vol. 25(5), pages 559-577, May.
    2. Graham Cookson & Simon Jones & Ioannis Laliotis, 2017. "Cancelled Procedures in the English NHS: Evidence from the 2010 Tariff Reform," Health Economics, John Wiley & Sons, Ltd., vol. 26(12), pages 126-139, December.
    3. Daniel Kuehnle & Christoph Wunder, 2017. "The Effects of Smoking Bans on Self‐Assessed Health: Evidence from Germany," Health Economics, John Wiley & Sons, Ltd., vol. 26(3), pages 321-337, March.
    4. Hajdu, Tamás & Hajdu, Gábor, 2018. "Smoking ban and health at birth: Evidence from Hungary," Economics & Human Biology, Elsevier, vol. 30(C), pages 37-47.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    2. Peter Hull & Michal Kolesár & Christopher Walters, 2022. "Labor by design: contributions of David Card, Joshua Angrist, and Guido Imbens," Scandinavian Journal of Economics, Wiley Blackwell, vol. 124(3), pages 603-645, July.
    3. Lechner, Michael, 2011. "The Estimation of Causal Effects by Difference-in-Difference Methods," Foundations and Trends(R) in Econometrics, now publishers, vol. 4(3), pages 165-224, November.
    4. van der Klaauw, Bas, 2014. "From micro data to causality: Forty years of empirical labor economics," Labour Economics, Elsevier, vol. 30(C), pages 88-97.
    5. Committee, Nobel Prize, 2021. "Answering causal questions using observational data," Nobel Prize in Economics documents 2021-2, Nobel Prize Committee.
    6. Daniel Kuehnle & Christoph Wunder, 2017. "The Effects of Smoking Bans on Self‐Assessed Health: Evidence from Germany," Health Economics, John Wiley & Sons, Ltd., vol. 26(3), pages 321-337, March.
    7. Rösner, Anja & Haucap, Justus & Heimeshoff, Ulrich, 2020. "The impact of consumer protection in the digital age: Evidence from the European Union," International Journal of Industrial Organization, Elsevier, vol. 73(C).
    8. Erlend E. Bø & Joel Slemrod & Thor O. Thoresen, 2015. "Taxes on the Internet: Deterrence Effects of Public Disclosure," American Economic Journal: Economic Policy, American Economic Association, vol. 7(1), pages 36-62, February.
    9. Sergei Guriev & Biagio Speciale & Michele Tuccio, 2019. "How do Regulated and Unregulated Labor Markets Respond to Shocks? Evidence from Immigrants During the Great Recession," The Journal of Law, Economics, and Organization, Oxford University Press, vol. 35(1), pages 37-76.
    10. Jones A.M & Rice N, 2009. "Econometric Evaluation of Health Policies," Health, Econometrics and Data Group (HEDG) Working Papers 09/09, HEDG, c/o Department of Economics, University of York.
    11. Herrero Prieto, Luis César, 2009. "La investigación en economía de la cultura en España: un estudio bibliométrico/Research in Cultural Economics in Spain: A Bibliometric Study," Estudios de Economia Aplicada, Estudios de Economia Aplicada, vol. 27, pages 35-62, Abril.
    12. Lewbel, Arthur & Yang, Thomas Tao, 2016. "Identifying the average treatment effect in ordered treatment models without unconfoundedness," Journal of Econometrics, Elsevier, vol. 195(1), pages 1-22.
    13. David Card, 2022. "Design-Based Research in Empirical Microeconomics," American Economic Review, American Economic Association, vol. 112(6), pages 1773-1781, June.
    14. Lin, Boqiang & Li, Xuehui, 2011. "The effect of carbon tax on per capita CO2 emissions," Energy Policy, Elsevier, vol. 39(9), pages 5137-5146, September.
    15. Jason Poulos & Andrea Albanese & Andrea Mercatanti & Fan Li, 2021. "Retrospective causal inference via matrix completion, with an evaluation of the effect of European integration on cross-border employment," Papers 2106.00788, arXiv.org.
    16. Susan Athey & Guido W. Imbens, 2006. "Identification and Inference in Nonlinear Difference-in-Differences Models," Econometrica, Econometric Society, vol. 74(2), pages 431-497, March.
    17. Athey, Susan & Imbens, Guido W., 2022. "Design-based analysis in Difference-In-Differences settings with staggered adoption," Journal of Econometrics, Elsevier, vol. 226(1), pages 62-79.
    18. Jeffrey Smith & Arthur Sweetman, 2016. "Viewpoint: Estimating the causal effects of policies and programs," Canadian Journal of Economics, Canadian Economics Association, vol. 49(3), pages 871-905, August.
    19. Sampaio, Breno, 2010. "On the identification of the effect of prohibiting hand-held cell phone use while driving: Comment," Transportation Research Part A: Policy and Practice, Elsevier, vol. 44(9), pages 766-770, November.
    20. Reimer, Matthew N. & Haynie, Alan C., 2018. "Mechanisms matter for evaluating the economic impacts of marine reserves," Journal of Environmental Economics and Management, Elsevier, vol. 88(C), pages 427-446.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:hlthec:v:21:y:2012:i:10:p:1234-1249. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www3.interscience.wiley.com/cgi-bin/jhome/5749 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.