IDEAS home Printed from https://ideas.repec.org/a/bla/jorssa/v172y2009i1p161-190.html
   My bibliography  Save this article

Hitting and missing targets by ambulance services for emergency calls: effects of different systems of performance measurement within the UK

Author

Listed:
  • Gwyn Bevan
  • Richard Hamblin

Abstract

Summary. Following devolution, differences developed between UK countries in systems of measuring performance against a common target that ambulance services ought to respond to 75% of calls for what may be immediately life threatening emergencies (category A calls) within 8 minutes. Only in England was this target integral to a ranking system of ‘star rating’, which inflicted reputational damage on services that failed to hit targets, and only in England has this target been met. In other countries, the target has been missed by such large margins that services would have been publicly reported as failing, if they had been covered by the English system of star ratings. The paper argues that this case‐study adds to evidence from comparisons of different systems of hospital performance measurement that, to have an effect, these systems need to be designed to inflict reputational damage on those that have performed poorly; and it explores implications of this hypothesis. The paper also asks questions about the adequacy of systems of performance measurement of ambulance services in UK countries.

Suggested Citation

  • Gwyn Bevan & Richard Hamblin, 2009. "Hitting and missing targets by ambulance services for emergency calls: effects of different systems of performance measurement within the UK," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 172(1), pages 161-190, January.
    • Handle: RePEc:bla:jorssa:v:172:y:2009:i:1:p:161-190
    DOI: 10.1111/j.1467-985X.2008.00557.x
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/j.1467-985X.2008.00557.x
    Download Restriction: no
    File URL: https://libkey.io/10.1111/j.1467-985X.2008.00557.x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Simon Burgess & Denise Gossage & Carol Propper, 2003. "Explaining differences in hospital performance: Does the answer lie in the labour market?," The Centre for Market and Public Organisation 03/091, The Centre for Market and Public Organisation, University of Bristol, UK.
    2. Peter Smith, 2005. "Performance Measurement in Health Care: History, Challenges and Prospects," Public Money & Management, Taylor & Francis Journals, vol. 25(4), pages 213-220.
    3. Kelman, Steven & Friedman, John N., 2007. "Performance Improvement and Performance Dysfunction: An Empirical Examination of Impacts of the Emergency Room Wait-Time Target in the English National Health Service," Working Paper Series rwp07-034, Harvard University, John F. Kennedy School of Government.
    4. David Dranove & Daniel Kessler & Mark McClellan & Mark Satterthwaite, 2003. "Is More Information Better? The Effects of "Report Cards" on Health Care Providers," Journal of Political Economy, University of Chicago Press, vol. 111(3), pages 555-588, June.
    5. Propper Carol & Sutton Matt & Whitnall Carolyn & Windmeijer Frank, 2008. "Did 'Targets and Terror' Reduce Waiting Times in England for Hospital Care?," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 8(2), pages 1-27, January.
    6. Rowena Jacobs & Maria Goddard, 2007. "How Do Performance Indicators Add Up? An Examination of Composite Indicators in Public Services," Public Money & Management, Taylor & Francis Journals, vol. 27(2), pages 103-110, April.
    7. D. J. Spiegelhalter, 1999. "Surgical audit: statistical lessons from Nightingale and Codman," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 162(1), pages 45-58.
    8. Gwyn Bevan, 2006. "Setting Targets for Health Care Performance," National Institute Economic Review, National Institute of Economic and Social Research, vol. 197(1), pages 67-79, July.
    9. Alvarez-Rosete, Arturo & Bevan, Gwyn & Mays, Nicholas & Dixon, Jennifer, 2005. "Effect of diverging policy across the NHS," LSE Research Online Documents on Economics 15513, London School of Economics and Political Science, LSE Library.
    10. Stevens, Philip Andrew & Stokes, Lucy & O'Mahony, Mary, 2006. "Metrics, Targets and Performance," National Institute Economic Review, National Institute of Economic and Social Research, vol. 197, pages 80-92, July.
    11. Le Grand, Julian, 2003. "Motivation, Agency, and Public Policy: Of Knights and Knaves, Pawns and Queens," OUP Catalogue, Oxford University Press, number 9780199266999.
    12. Rowena Jacobs & Peter Smith, 2004. "A descriptive analysis of general acute Trust star ratings," Working Papers 189chedp, Centre for Health Economics, University of York.
    13. Sheila M. Bird & Cox Sir David & Vern T. Farewell & Goldstein Harvey & Holt Tim & Smith Peter C., 2005. "Performance indicators: good, bad, and ugly," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 168(1), pages 1-27, January.
    14. Katharina Hauck & Andrew Street, 2007. "Do targets matter? A comparison of English and Welsh National Health priorities," Health Economics, John Wiley & Sons, Ltd., vol. 16(3), pages 275-290, March.
    15. Harvey Goldstein & David J. Spiegelhalter, 1996. "League Tables and Their Limitations: Statistical Issues in Comparisons of Institutional Performance," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 159(3), pages 385-409, May.
    16. Bevan, Gwyn & Cornwell, Jocelyn, 2006. "Structure and logic of regulation and governance of quality of health care: was OFSTED a model for the Commission for Health Improvement?," Health Economics, Policy and Law, Cambridge University Press, vol. 1(4), pages 343-370, October.
    17. Patel, Brijesh & Chaussalet, Thierry & Millard, Peter, 2008. "Balancing the NHS balanced scorecard!," European Journal of Operational Research, Elsevier, vol. 185(3), pages 905-914, March.
    18. Geoffrey Heath & James Radcliffe, 2007. "Performance Measurement and the English Ambulance Service," Public Money & Management, Taylor & Francis Journals, vol. 27(3), pages 223-228, June.
    19. Friedman, John N. & Kelman, Steven, 2007. "Effort as Investment: Analyzing the Response to Incentives," Working Paper Series rwp07-024, Harvard University, John F. Kennedy School of Government.
    20. Christopher Hood, 2007. "Public Service Management by Numbers: Why Does it Vary? Where Has it Come From? What Are the Gaps and the Puzzles?," Public Money & Management, Taylor & Francis Journals, vol. 27(2), pages 95-102, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Blog mentions

    As found by EconAcademics.org, the blog aggregator for Economics research:
    1. Incorporating reputational concerns in public sector reform: it may be effective but needs creative monitoring
      by Jed Friedman in Development Impact on 2012-06-06 19:02:57

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Schang, Laura & Hynninen, Yrjänä & Morton, Alec & Salo, Ahti, 2016. "Developing robust composite measures of healthcare quality – Ranking intervals and dominance relations for Scottish Health Boards," Social Science & Medicine, Elsevier, vol. 162(C), pages 59-67.
    2. Burgess, Simon & Wilson, Deborah & Worth, Jack, 2013. "A natural experiment in school accountability: The impact of school performance information on pupil progress," Journal of Public Economics, Elsevier, vol. 106(C), pages 57-67.
    3. Carl Deschamps & Jan Mattijs, 2015. "Anatomy of a performance management system: the elusive path from targets to productivity," Working Papers CEB 15-037, ULB -- Universite Libre de Bruxelles.
    4. Bevan, Gwyn & Evans, Alice & Nuti, Sabina, 2018. "Reputations count: why benchmarking performance is improving health care across the world," LSE Research Online Documents on Economics 86469, London School of Economics and Political Science, LSE Library.
    5. Thomas Bolognesi & Géraldine Pflieger, 2021. "In the shadow of sunshine regulation: Explaining disclosure biases," Regulation & Governance, John Wiley & Sons, vol. 15(1), pages 200-225, January.
    6. Knight, V.A. & Harper, P.R. & Smith, L., 2012. "Ambulance allocation for maximal survival with heterogeneous outcome measures," Omega, Elsevier, vol. 40(6), pages 918-926.
    7. Carol Propper & Deborah Wilson, 2012. "The Use of Performance Measures in Health Care Systems," Chapters, in: Andrew M. Jones (ed.), The Elgar Companion to Health Economics, Second Edition, chapter 33, Edward Elgar Publishing.
    8. Pereira, Miguel Alves & Marques, Rui Cunha, 2022. "Is sunshine regulation the new prescription to brighten up public hospitals in Portugal?," Socio-Economic Planning Sciences, Elsevier, vol. 84(C).
    9. Kristian Niemietz, 2015. "Internal Markets, Management by Targets, and Quasi-Markets: An Analysis of Health Care Reforms in the English NHS," Economic Affairs, Wiley Blackwell, vol. 35(1), pages 93-108, February.
    10. Propper, Carol & Sutton, Matt & Whitnall, Carolyn & Windmeijer, Frank, 2010. "Incentives and targets in hospital care: Evidence from a natural experiment," Journal of Public Economics, Elsevier, vol. 94(3-4), pages 318-335, April.
    11. van de Ven, Wynand P.M.M. & Beck, Konstantin & Buchner, Florian & Schokkaert, Erik & Schut, F.T. (Erik) & Shmueli, Amir & Wasem, Juergen, 2013. "Preconditions for efficiency and affordability in competitive healthcare markets: Are they fulfilled in Belgium, Germany, Israel, the Netherlands and Switzerland?," Health Policy, Elsevier, vol. 109(3), pages 226-245.
    12. De Rosis, Sabina & Guidotti, Elisa & Zuccarino, Sara & Venturi, Giulia & Ferré, Francesca, 2020. "Waiting time information in the Italian NHS: A citizen perspective," Health Policy, Elsevier, vol. 124(8), pages 796-804.
    13. Nick Manning & Joanna Watkins, 2013. "Targeting Results, Diagnosing the Means," World Bank Publications - Reports 25488, The World Bank Group.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Besley, Timothy & Burchardi, Konrad B. & Bevan, Gwen, 2009. "Naming and shaming: the impacts of different regimes on hospital waiting times in England and Wales," LSE Research Online Documents on Economics 33775, London School of Economics and Political Science, LSE Library.
    2. Besley, Tim & Bevan, Gwyn & Burchardi, Konrad, 2009. "Naming & Shaming: The impacts of different regimes on hospital waiting times in England and Wales," CEPR Discussion Papers 7306, C.E.P.R. Discussion Papers.
    3. Carol Propper & Deborah Wilson, 2012. "The Use of Performance Measures in Health Care Systems," Chapters, in: Andrew M. Jones (ed.), The Elgar Companion to Health Economics, Second Edition, chapter 33, Edward Elgar Publishing.
    4. Propper, Carol & Sutton, Matt & Whitnall, Carolyn & Windmeijer, Frank, 2010. "Incentives and targets in hospital care: Evidence from a natural experiment," Journal of Public Economics, Elsevier, vol. 94(3-4), pages 318-335, April.
    5. Shimaa Elkomy & Graham Cookson, 2020. "Performance Management Strategy: Waiting Time in the English National Health Services," Public Organization Review, Springer, vol. 20(1), pages 95-112, March.
    6. Matilde P. Machado & Ricardo Mora & Antonio Romero-Medina, 2012. "Can We Infer Hospital Quality From Medical Graduates’ Residency Choices?," Journal of the European Economic Association, European Economic Association, vol. 10(6), pages 1400-1424, December.
    7. Breton, Mylaine & Smithman, Mélanie Ann & Sasseville, Martin & Kreindler, Sara A. & Sutherland, Jason M. & Beauséjour, Marie & Green, Michael & Marshall, Emily Gard & Jbilou, Jalila & Shaw, Jay & Brou, 2020. "How the design and implementation of centralized waiting lists influence their use and effect on access to healthcare - A realist review," Health Policy, Elsevier, vol. 124(8), pages 787-795.
    8. Martin Gaynor & Carol Propper & Stephan Seiler, 2016. "Free to Choose? Reform, Choice, and Consideration Sets in the English National Health Service," American Economic Review, American Economic Association, vol. 106(11), pages 3521-3557, November.
    9. repec:zbw:rwirep:0331 is not listed on IDEAS
    10. Martin Gaynor & Carol Propper & Stephan Seiler, 2012. "Free to Choose? Reform and Demand Response in the English National Health Service," The Centre for Market and Public Organisation 12/297, The Centre for Market and Public Organisation, University of Bristol, UK.
    11. George Leckie & Harvey Goldstein, 2009. "The limitations of using school league tables to inform school choice," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 172(4), pages 835-851, October.
    12. Morton, Alec & Bevan, Gwyn, 2008. "What's in a wait: Contrasting management science and economic perspectives on waiting for emergency care," Health Policy, Elsevier, vol. 85(2), pages 207-217, February.
    13. Isabella Sulis & Mariano Porcu, 2015. "Assessing Divergences in Mathematics and Reading Achievement in Italian Primary Schools: A Proposal of Adjusted Indicators of School Effectiveness," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 122(2), pages 607-634, June.
    14. Vainieri, Milena & Vola, Federico & Gomez Soriano, Gregorio & Nuti, Sabina, 2016. "How to set challenging goals and conduct fair evaluation in regional public health systems. Insights from Valencia and Tuscany Regions," Health Policy, Elsevier, vol. 120(11), pages 1270-1278.
    15. Giuseppina Guagnano & Maria Rita Sebastiani, 2018. "Away from Dissatisfaction, Closer to Well-Being: A Multidimensional Synthetic Measure," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 136(3), pages 977-997, April.
    16. David I. Ohlssen & Linda D. Sharples & David J. Spiegelhalter, 2007. "A hierarchical modelling framework for identifying unusual performance in health care providers," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 170(4), pages 865-890, October.
    17. Sylvia Richardson, 2022. "Statistics in times of increasing uncertainty," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 185(4), pages 1471-1496, October.
    18. Juliette Malley & José‐Luis Fernández, 2010. "Measuring Quality In Social Care Services: Theory And Practice," Annals of Public and Cooperative Economics, Wiley Blackwell, vol. 81(4), pages 559-582, December.
    19. Nazarov, Vladimir (Назаров, Владимир) & Davis, Christopher Mark (Дэвис, Кристофер Марк) & Gerry, Christopher J. (Гэрри, Кристофер Дж.) & Polyakova, Aleksandra (Полякова, Александра) & Sisigina, Natali, 2015. "Evaluating the Effectiveness and Efficiency of the Health System [Оценка Эффективности И Результативности Системы Здравоохранения]," Published Papers mn41, Russian Presidential Academy of National Economy and Public Administration.
    20. Nadja Kairies, 2012. "P4P, Reputation and the Reduction of Costly Overprovision," Ruhr Economic Papers 0331, Rheinisch-Westfälisches Institut für Wirtschaftsforschung, Ruhr-Universität Bochum, Universität Dortmund, Universität Duisburg-Essen.
    21. Eric French & Elaine Kelly & Richard Cookson & Carol Propper & Miqdad Asaria & Rosalind Raine, 2016. "Socio‐Economic Inequalities in Health Care in England," Fiscal Studies, Institute for Fiscal Studies, vol. 37, pages 371-403, September.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jorssa:v:172:y:2009:i:1:p:161-190. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://edirc.repec.org/data/rssssea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.