IDEAS home Printed from https://ideas.repec.org/a/wly/fufsci/v6y2024i1ne172.html
   My bibliography  Save this article

Quality indicators for Delphi studies

Author

Listed:
  • Jon Landeta
  • Aitziber Lertxundi

Abstract

The Delphi method is a technique of social research that seeks to obtain a reliable group opinion from experts. It was first created for military purposes in the mid‐1950s. Since then, its use in the scientific field has continued to spread to different disciplines and aims. Despite this expansion, however, not set of indicators of the quality of Delphi studies has yet to be developed that might provide the reader—whether an expert in the technique or not—with some framework of reference whereby to gauge what credibility should be afforded to the results of the study. In this paper, following a thorough review of the literature on the criteria used to assess Delphi studies and the items of evaluation recommended for inclusion in Delphi reports, we determine what characteristics a quality evaluation indicator for this technique should have and propose a battery of indicators based on these characteristics, which should for preference be included in the final report of a Delphi study. The proposed indicators focus on three areas that are particularly relevant to the quality of Delphi research: the quality of the panel of participating experts, the way in which relevant information is obtained from the experts, and the quality of the interaction generated among the experts.

Suggested Citation

  • Jon Landeta & Aitziber Lertxundi, 2024. "Quality indicators for Delphi studies," Futures & Foresight Science, John Wiley & Sons, vol. 6(1), March.
    • Handle: RePEc:wly:fufsci:v:6:y:2024:i:1:n:e172
    DOI: 10.1002/ffo2.172
    as

    Download full text from publisher

    File URL: https://doi.org/10.1002/ffo2.172
    Download Restriction: no
    File URL: https://libkey.io/10.1002/ffo2.172?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Landeta, Jon & Barrutia, Jon, 2011. "People consultation to construct the future: A Delphi application," International Journal of Forecasting, Elsevier, vol. 27(1), pages 134-151, January.
    2. Barrios, Maite & Guilera, Georgina & Nuño, Laura & Gómez-Benito, Juana, 2021. "Consensus in the delphi method: What makes a decision change?," Technological Forecasting and Social Change, Elsevier, vol. 163(C).
    3. Cynthia P. Iglesias & Alexander Thompson & Wolf H. Rogowski & Katherine Payne, 2016. "Reporting Guidelines for the Use of Expert Judgement in Model-Based Economic Evaluations," PharmacoEconomics, Springer, vol. 34(11), pages 1161-1172, November.
    4. Landeta, Jon & Barrutia, Jon, 2011. "People consultation to construct the future: A Delphi application," International Journal of Forecasting, Elsevier, vol. 27(1), pages 134-151.
    5. Mauksch, Stefanie & von der Gracht, Heiko A. & Gordon, Theodore J., 2020. "Who is an expert for foresight? A review of identification methods," Technological Forecasting and Social Change, Elsevier, vol. 154(C).
    6. Olaf Helmer & Nicholas Rescher, 1959. "On the Epistemology of the Inexact Sciences," Management Science, INFORMS, vol. 6(1), pages 25-52, October.
    7. Norman Dalkey & Olaf Helmer, 1963. "An Experimental Application of the DELPHI Method to the Use of Experts," Management Science, INFORMS, vol. 9(3), pages 458-467, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Alyami, Saleh. H. & Rezgui, Yacine & Kwan, Alan, 2013. "Developing sustainable building assessment scheme for Saudi Arabia: Delphi consultation approach," Renewable and Sustainable Energy Reviews, Elsevier, vol. 27(C), pages 43-54.
    2. Aldossary, Naief A. & Rezgui, Yacine & Kwan, Alan, 2015. "Consensus-based low carbon domestic design framework for sustainable homes," Renewable and Sustainable Energy Reviews, Elsevier, vol. 51(C), pages 417-432.
    3. Förster, Bernadette & von der Gracht, Heiko, 2014. "Assessing Delphi panel composition for strategic foresight — A comparison of panels based on company-internal and external participants," Technological Forecasting and Social Change, Elsevier, vol. 84(C), pages 215-229.
    4. Meijering, Jurian V. & Tobi, Hilde, 2016. "The effect of controlled opinion feedback on Delphi features: Mixed messages from a real-world Delphi experiment," Technological Forecasting and Social Change, Elsevier, vol. 103(C), pages 166-173.
    5. Engelke, Henning & Mauksch, Stefanie & Darkow, Inga-Lena & von der Gracht, Heiko A., 2015. "Opportunities for social enterprise in Germany — Evidence from an expert survey," Technological Forecasting and Social Change, Elsevier, vol. 90(PB), pages 635-646.
    6. Alcon, Francisco & Tapsuwan, Sorada & Martínez-Paz, José M. & Brouwer, Roy & de Miguel, María D., 2014. "Forecasting deficit irrigation adoption using a mixed stakeholder assessment methodology," Technological Forecasting and Social Change, Elsevier, vol. 83(C), pages 183-193.
    7. Di Zio, Simone & Bolzan, Mario & Marozzi, Marco, 2021. "Classification of Delphi outputs through robust ranking and fuzzy clustering for Delphi-based scenarios," Technological Forecasting and Social Change, Elsevier, vol. 173(C).
    8. Christoph Markmann & Alexander Spickermann & Heiko A. von der Gracht & Alexander Brem, 2021. "Improving the question formulation in Delphi‐like surveys: Analysis of the effects of abstract language and amount of information on response behavior," Futures & Foresight Science, John Wiley & Sons, vol. 3(1), March.
    9. Kattirtzi, Michael & Winskel, Mark, 2020. "When experts disagree: Using the Policy Delphi method to analyse divergent expert expectations and preferences on UK energy futures," Technological Forecasting and Social Change, Elsevier, vol. 153(C).
    10. Isabela Caroline de Sousa & Tiago F. A. C. Sigahi & Izabela Simon Rampasso & Gustavo Hermínio Salati Marcondes de Moraes & Walter Leal Filho & João Henrique Paulino Pires Eustachio & Rosley Anholon, 2024. "A Delphi–Fuzzy Delphi Study on SDGs 9 and 12 after COVID-19: Case Study in Brazil," Forecasting, MDPI, vol. 6(3), pages 1-18, July.
    11. Horrillo, A. & Escribano, M. & Mesias, F.J. & Elghannam, A. & Gaspar, P., 2016. "Is there a future for organic production in high ecological value ecosystems?," Agricultural Systems, Elsevier, vol. 143(C), pages 114-125.
    12. Belton, Ian & MacDonald, Alice & Wright, George & Hamlin, Iain, 2019. "Improving the practical application of the Delphi method in group-based judgment: A six-step prescription for a well-founded and defensible process," Technological Forecasting and Social Change, Elsevier, vol. 147(C), pages 72-82.
    13. Sebastian Hinderer & Leif Brändle & Andreas Kuckertz, 2021. "Transition to a Sustainable Bioeconomy," Sustainability, MDPI, vol. 13(15), pages 1-16, July.
    14. Barrios, Maite & Guilera, Georgina & Nuño, Laura & Gómez-Benito, Juana, 2021. "Consensus in the delphi method: What makes a decision change?," Technological Forecasting and Social Change, Elsevier, vol. 163(C).
    15. Wright, George & Rowe, Gene, 2011. "Group-based judgmental forecasting: An integration of extant knowledge and the development of priorities for a new research agenda," International Journal of Forecasting, Elsevier, vol. 27(1), pages 1-13, January.
    16. Winkler, Jens & Kuklinski, Christian Paul Jian-Wei & Moser, Roger, 2015. "Decision making in emerging markets: The Delphi approach's contribution to coping with uncertainty and equivocality," Journal of Business Research, Elsevier, vol. 68(5), pages 1118-1126.
    17. Spickermann, Alexander & Zimmermann, Martin & von der Gracht, Heiko A., 2014. "Surface- and deep-level diversity in panel selection — Exploring diversity effects on response behaviour in foresight," Technological Forecasting and Social Change, Elsevier, vol. 85(C), pages 105-120.
    18. Brown, Philip & Daigneault, Adam, 2015. "Managing the Invasive Small Indian Mongoose in Fiji," Agricultural and Resource Economics Review, Northeastern Agricultural and Resource Economics Association, vol. 44(3), pages 1-16, December.
    19. repec:eee:intfor:v:27:y:2011:i:1:p:1-13 is not listed on IDEAS
    20. Marco van Gelderen & Johan Wiklund & Jeffery S. McMullen, 2021. "Entrepreneurship in the Future: A Delphi Study of ETP and JBV Editorial Board Members," Entrepreneurship Theory and Practice, , vol. 45(5), pages 1239-1275, September.
    21. Winkler, Jens & Moser, Roger, 2016. "Biases in future-oriented Delphi studies: A cognitive perspective," Technological Forecasting and Social Change, Elsevier, vol. 105(C), pages 63-76.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:fufsci:v:6:y:2024:i:1:n:e172. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://doi.org/10.1002/(ISSN)2573-5152 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.