IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0020476.html
   My bibliography  Save this article

Using and Reporting the Delphi Method for Selecting Healthcare Quality Indicators: A Systematic Review

Author

Listed:
  • Rym Boulkedid
  • Hendy Abdoul
  • Marine Loustau
  • Olivier Sibony
  • Corinne Alberti

Abstract

Objective: Delphi technique is a structured process commonly used to developed healthcare quality indicators, but there is a little recommendation for researchers who wish to use it. This study aimed 1) to describe reporting of the Delphi method to develop quality indicators, 2) to discuss specific methodological skills for quality indicators selection 3) to give guidance about this practice. Methodology and Main Finding: Three electronic data bases were searched over a 30 years period (1978–2009). All articles that used the Delphi method to select quality indicators were identified. A standardized data extraction form was developed. Four domains (questionnaire preparation, expert panel, progress of the survey and Delphi results) were assessed. Of 80 included studies, quality of reporting varied significantly between items (9% for year's number of experience of the experts to 98% for the type of Delphi used). Reporting of methodological aspects needed to evaluate the reliability of the survey was insufficient: only 39% (31/80) of studies reported response rates for all rounds, 60% (48/80) that feedback was given between rounds, 77% (62/80) the method used to achieve consensus and 57% (48/80) listed quality indicators selected at the end of the survey. A modified Delphi procedure was used in 49/78 (63%) with a physical meeting of the panel members, usually between Delphi rounds. Median number of panel members was 17(Q1:11; Q3:31). In 40/70 (57%) studies, the panel included multiple stakeholders, who were healthcare professionals in 95% (38/40) of cases. Among 75 studies describing criteria to select quality indicators, 28 (37%) used validity and 17(23%) feasibility. Conclusion: The use and reporting of the Delphi method for quality indicators selection need to be improved. We provide some guidance to the investigators to improve the using and reporting of the method in future surveys.

Suggested Citation

  • Rym Boulkedid & Hendy Abdoul & Marine Loustau & Olivier Sibony & Corinne Alberti, 2011. "Using and Reporting the Delphi Method for Selecting Healthcare Quality Indicators: A Systematic Review," PLOS ONE, Public Library of Science, vol. 6(6), pages 1-9, June.
    • Handle: RePEc:plo:pone00:0020476
    DOI: 10.1371/journal.pone.0020476
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0020476
    Download Restriction: no
    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0020476&type=printable
    Download Restriction: no
    File URL: https://libkey.io/10.1371/journal.pone.0020476?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Fink, A. & Kosecoff, J. & Chassin, M. & Brook, R.H., 1984. "Consensus methods: Characteristics and guidelines for use," American Journal of Public Health, American Public Health Association, vol. 74(9), pages 979-983.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Melanie Kleynen & Susy M Braun & Michel H Bleijlevens & Monique A Lexis & Sascha M Rasquin & Jos Halfens & Mark R Wilson & Anna J Beurskens & Rich S W Masters, 2014. "Using a Delphi Technique to Seek Consensus Regarding Definitions, Descriptions and Classification of Terms Related to Implicit and Explicit Forms of Motor Learning," PLOS ONE, Public Library of Science, vol. 9(6), pages 1-11, June.
    2. Miranda L van Hooff & Jan van Loon & Jacques van Limbeek & Marinus de Kleuver, 2014. "The Nijmegen Decision Tool for Chronic Low Back Pain. Development of a Clinical Decision Tool for Secondary or Tertiary Spine Care Specialists," PLOS ONE, Public Library of Science, vol. 9(8), pages 1-12, August.
    3. Tracey Smythe & Andrew Wainwright & Allen Foster & Christopher Lavy, 2017. "What is a good result after clubfoot treatment? A Delphi-based consensus on success by regional clubfoot trainers from across Africa," PLOS ONE, Public Library of Science, vol. 12(12), pages 1-9, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Shannon Li & Anne Honey & Francesca Coniglio & Peter Schaecken, 2022. "Mental Health Peer Worker Perspectives on Resources Developed from Lived Experience Research Findings: A Delphi Study," IJERPH, MDPI, vol. 19(7), pages 1-15, March.
    2. Dana Hübelová & Martina Kuncová & Hana Vojáčková & Jitka Coufalová & Alice Kozumplíková & Francois Stefanus Lategan & Beatrice-Elena Chromková Manea, 2021. "Inequalities in Health: Methodological Approaches to Spatial Differentiation," IJERPH, MDPI, vol. 18(23), pages 1-21, November.
    3. West, Allison & Duggan, Anne K. & Gruss, Kelsey & Minkovitz, Cynthia S., 2018. "Creating a measurement framework for service coordination in maternal and early childhood home visiting: An evidence-informed, expert process," Children and Youth Services Review, Elsevier, vol. 89(C), pages 289-297.
    4. Sundmacher, Leonie & Fischbach, Diana & Schuettig, Wiebke & Naumann, Christoph & Augustin, Uta & Faisst, Cristina, 2015. "Which hospitalisations are ambulatory care-sensitive, to what degree, and how could the rates be reduced? Results of a group consensus study in Germany," Health Policy, Elsevier, vol. 119(11), pages 1415-1423.
    5. Gabel, Matthew J. & Shipan, Charles R., 2004. "A social choice approach to expert consensus panels," Journal of Health Economics, Elsevier, vol. 23(3), pages 543-564, May.
    6. Mitra Hannani & Marc Bascompta & Mojtaba Gerami Sabzevar & Hesam Dehghani & Ali Asghar Khajevandi, 2023. "Causal Analysis of Safety Risk Perception of Iranian Coal Mining Workers Using Fuzzy Delphi and DEMATEL," Sustainability, MDPI, vol. 15(19), pages 1-19, September.
    7. Inmaculada Serrano & Mercedes Fernández & Eva Bajo Marcos, 2024. "Building a Set of Indicators to Assess Migrant Children's Integration in Europe: A Co-Creation Approach," Child Indicators Research, Springer;The International Society of Child Indicators (ISCI), vol. 17(6), pages 2389-2417, December.
    8. Sarah G. McCarthy & Richard R. Rushforth, 2025. "Identifying Barriers to Implementation of Regenerative Agricultural Solutions Through Convergence Research," Land, MDPI, vol. 14(3), pages 1-17, February.
    9. Michael A Rubin & Makoto Jones & Molly Leecaster & Karim Khader & Willy Ray & Angela Huttner & Benedikt Huttner & Damon Toth & Theodore Sablay & Robert J Borotkanics & Dale N Gerding & Matthew H Samor, 2013. "A Simulation-Based Assessment of Strategies to Control Clostridium Difficile Transmission and Infection," PLOS ONE, Public Library of Science, vol. 8(11), pages 1-11, November.
    10. Otto H. Swank & Bauke Visser, 2009. "Decision Making and Learning in a Globalizing World," Economics Working Papers ECO2009/20, European University Institute.
    11. Marzieh Fallah & Lanndon Ocampo, 2021. "The use of the Delphi method with non-parametric analysis for identifying sustainability criteria and indicators in evaluating ecotourism management: the case of Penang National Park (Malaysia)," Environment Systems and Decisions, Springer, vol. 41(1), pages 45-62, March.
    12. Violaine Smaïl-Faugeron & Hélène Fron Chabouis & Pierre Durieux & Jean-Pierre Attal & Michèle Muller-Bolla & Frédéric Courson, 2013. "Development of a Core Set of Outcomes for Randomized Controlled Trials with Multiple Outcomes – Example of Pulp Treatments of Primary Teeth for Extensive Decay in Children," PLOS ONE, Public Library of Science, vol. 8(1), pages 1-9, January.
    13. Ravonne A. Green, 2014. "The Delphi Technique in Educational Research," SAGE Open, , vol. 4(2), pages 21582440145, April.
    14. Sharon Barak & Tzlil Rabinovitz & Achinoam Ben Akiva-Maliniak & Rony Schenker & Lian Meiry & Riki Tesler, 2022. "An Individually Tailored Program to Increase Healthy Lifestyle Behaviors among the Elderly," IJERPH, MDPI, vol. 19(17), pages 1-12, September.
    15. Justus N Agumba, 2015. "Validating and Identifying Health and Safety Performance Improvement Indicators: Experience of Using Delphi Technique," Journal of Economics and Behavioral Studies, AMH International, vol. 7(3), pages 14-22.
    16. Shani, Segev & Siebzehner, Miriam Ines & Luxenburg, Osnat & Shemer, Joshua, 2000. "Setting priorities for the adoption of health technologies on a national level -- the Israeli experience," Health Policy, Elsevier, vol. 54(3), pages 169-185, December.
    17. Sarvari, Hadi & Baghbaderani, Alireza Babaie & Chan, Daniel W.M. & Beer, Michael, 2024. "Determining the significant contributing factors to the occurrence of human errors in the urban construction projects: A Delphi-SWARA study approach," Technological Forecasting and Social Change, Elsevier, vol. 205(C).
    18. Otto H. Swank & Bauke Visser, 2015. "Learning from Others? Decision Rights, Strategic Communication, and Reputational Concerns," American Economic Journal: Microeconomics, American Economic Association, vol. 7(4), pages 109-149, November.
    19. Johanna Schönrock-Adema & Maartje Visscher & A N Janet Raat & Paul L P Brand, 2015. "Development and Validation of the Scan of Postgraduate Educational Environment Domains (SPEED): A Brief Instrument to Assess the Educational Environment in Postgraduate Medical Education," PLOS ONE, Public Library of Science, vol. 10(9), pages 1-12, September.
    20. Caleb Chidozie Chinedu & Atif Saleem & Wan Hanim Nadrah Wan Muda, 2023. "Teaching and Learning Approaches: Curriculum Framework for Sustainability Literacy for Technical and Vocational Teacher Training Programmes in Malaysia," Sustainability, MDPI, vol. 15(3), pages 1-24, January.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0020476. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.