IDEAS home Printed from https://ideas.repec.org/a/eee/tefoso/v170y2021ics0040162521003292.html
   My bibliography  Save this article

Delphi with feedback of rationales: How large can a Delphi group be such that participants are not overloaded, de-motivated, or disengaged?

Author

Listed:
  • Belton, Ian
  • Wright, George
  • Sissons, Aileen
  • Bolger, Fergus
  • Crawford, Megan M.
  • Hamlin, Iain
  • Taylor Browne Lūka, Courtney
  • Vasilichi, Alexandrina

Abstract

In this paper, we investigate the effect of Delphi group size and opinion diversity on group members’ information load as well as on their overall experience of the Delphi process - in terms of task involvement (enjoyment and interest) and in terms of group sway (the influence and helpfulness of others’ rationales). For Delphi applications involving the exchange of rationales between participants, we found no evidence that group sizes of up to 19 participants cause information overload or de-motivation and disengagement of participants.

Suggested Citation

  • Belton, Ian & Wright, George & Sissons, Aileen & Bolger, Fergus & Crawford, Megan M. & Hamlin, Iain & Taylor Browne Lūka, Courtney & Vasilichi, Alexandrina, 2021. "Delphi with feedback of rationales: How large can a Delphi group be such that participants are not overloaded, de-motivated, or disengaged?," Technological Forecasting and Social Change, Elsevier, vol. 170(C).
    • Handle: RePEc:eee:tefoso:v:170:y:2021:i:c:s0040162521003292
    DOI: 10.1016/j.techfore.2021.120897
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0040162521003292
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.techfore.2021.120897?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. David M. Boje & J. Keith Murnighan, 1982. "Group Confidence Pressures in Iterative Decisions," Management Science, INFORMS, vol. 28(10), pages 1187-1196, October.
    2. Bolger, Fergus & Rowe, Gene & Belton, Ian & Crawford, Megan M & Hamlin, Iain & Sissons, Aileen & Taylor Browne Lūka, Courtney & Vasilichi, Alexandrina & Wright, George, 2020. "The Simulated Group Response Paradigm: A new approach to the study of opinion change in Delphi and other structured-group techniques," OSF Preprints 4ufzg, Center for Open Science.
    3. de Loë, Rob C. & Melnychuk, Natalya & Murray, Dan & Plummer, Ryan, 2016. "Advancing the State of Policy Delphi Practice: A Systematic Review Evaluating Methodological Evolution, Innovation, and Opportunities," Technological Forecasting and Social Change, Elsevier, vol. 104(C), pages 78-88.
    4. Rajagopal, 2014. "The Human Factors," Palgrave Macmillan Books, in: Architecting Enterprise, chapter 9, pages 225-249, Palgrave Macmillan.
    5. Park, Jeong-Yeol & Jang, SooCheong (Shawn), 2013. "Confused by too many choices? Choice overload in tourism," Tourism Management, Elsevier, vol. 35(C), pages 1-12.
    6. Moonjung Choi & Han-Lim Choi, 2015. "Foresight for Science and Technology Priority Setting in Korea," Foresight-Russia Форсайт, CyberLeninka;Федеральное государственное автономное образовательное учреждение высшего образования «Национальный исследовательский университет «Высшая школа экономики», vol. 9(3 (eng)), pages 54-65.
    7. Lee Cronbach, 1951. "Coefficient alpha and the internal structure of tests," Psychometrika, Springer;The Psychometric Society, vol. 16(3), pages 297-334, September.
    8. Belton, Ian & MacDonald, Alice & Wright, George & Hamlin, Iain, 2019. "Improving the practical application of the Delphi method in group-based judgment: A six-step prescription for a well-founded and defensible process," Technological Forecasting and Social Change, Elsevier, vol. 147(C), pages 72-82.
    9. Aengenheyster, Stefan & Cuhls, Kerstin & Gerhold, Lars & Heiskanen-Schüttler, Maria & Huck, Jana & Muszynska, Monika, 2017. "Real-Time Delphi in practice — A comparative analysis of existing software-based tools," Technological Forecasting and Social Change, Elsevier, vol. 118(C), pages 15-27.
    10. Moonjung Choi & Han-Lim Choi, 2015. "Foresight for Science and Technology Priority Setting in Korea," Foresight and STI Governance (Foresight-Russia till No. 3/2015), National Research University Higher School of Economics, vol. 9(3), pages 54-65.
    11. Meijering, Jurian V. & Tobi, Hilde, 2016. "The effect of controlled opinion feedback on Delphi features: Mixed messages from a real-world Delphi experiment," Technological Forecasting and Social Change, Elsevier, vol. 103(C), pages 166-173.
    12. Iselin, Errol R., 1988. "The effects of information load and information diversity on decision quality in a structured decision task," Accounting, Organizations and Society, Elsevier, vol. 13(2), pages 147-164, March.
    13. Stanley F. Biggs & Jean C. Bedard & Brian G. Gaber & Thomas J. Linsmeier, 1985. "The Effects of Task Size and Similarity on the Decision Behavior of Bank Loan Officers," Management Science, INFORMS, vol. 31(8), pages 970-987, August.
    14. Norman Dalkey & Olaf Helmer, 1963. "An Experimental Application of the DELPHI Method to the Use of Experts," Management Science, INFORMS, vol. 9(3), pages 458-467, April.
    15. Fergus Bolger, 2018. "The Selection of Experts for (Probabilistic) Expert Knowledge Elicitation," International Series in Operations Research & Management Science, in: Luis C. Dias & Alec Morton & John Quigley (ed.), Elicitation, chapter 0, pages 393-443, Springer.
    16. Li, Na & Chen, Kaihua & Kou, Mingting, 2017. "Technology foresight in China: Academic studies, governmental practices and policy applications," Technological Forecasting and Social Change, Elsevier, vol. 119(C), pages 246-255.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Akartuna, Eray Arda & Johnson, Shane D. & Thornton, Amy, 2022. "Preventing the money laundering and terrorist financing risks of emerging technologies: An international policy Delphi study," Technological Forecasting and Social Change, Elsevier, vol. 179(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Belton, Ian & MacDonald, Alice & Wright, George & Hamlin, Iain, 2019. "Improving the practical application of the Delphi method in group-based judgment: A six-step prescription for a well-founded and defensible process," Technological Forecasting and Social Change, Elsevier, vol. 147(C), pages 72-82.
    2. Sokolov, Alexander & Shashnov, Sergey & Kotsemir, Maxim & Grebenyuk, Anna, 2019. "Quantitative analysis for a better-focused international STI collaboration policy: A case of BRICS," Technological Forecasting and Social Change, Elsevier, vol. 147(C), pages 221-242.
    3. Chang, Victor & Liu, Ben S.C. & Sudharshan, D. & Xu, Qianwen Ariel, 2021. "Towards an effective negotiation modeling: Investigating transboundary disputes with cases of lower possibilities," Technological Forecasting and Social Change, Elsevier, vol. 165(C).
    4. Bolger, Fergus & Rowe, Gene & Belton, Ian & Crawford, Megan M & Hamlin, Iain & Sissons, Aileen & Taylor Browne Lūka, Courtney & Vasilichi, Alexandrina & Wright, George, 2020. "The Simulated Group Response Paradigm: A new approach to the study of opinion change in Delphi and other structured-group techniques," OSF Preprints 4ufzg, Center for Open Science.
    5. XiaoJuan Zhang & Xiang Jinpeng & Farhan Khan, 2020. "The Influence of Social Media on Employee’s Knowledge Sharing Motivation: A Two-Factor Theory Perspective," SAGE Open, , vol. 10(3), pages 21582440209, July.
    6. Akartuna, Eray Arda & Johnson, Shane D. & Thornton, Amy, 2022. "Preventing the money laundering and terrorist financing risks of emerging technologies: An international policy Delphi study," Technological Forecasting and Social Change, Elsevier, vol. 179(C).
    7. Surapree Maolikul & Thira Chavarnakul & Somchai Kiatgamolchai, 2019. "Market Opportunity Analysis in Thailand: Case of Individual Power Sources by Thermoelectric-Generator Technology for Portable Electronics," International Journal of Innovation and Technology Management (IJITM), World Scientific Publishing Co. Pte. Ltd., vol. 16(03), pages 1-24, May.
    8. Chen Wei, 2021. "The influence of Consumers’ Purchase intention on Smart Wearable Device: A study of Consumers in East China," International Journal of Science and Business, IJSAB International, vol. 5(8), pages 46-72.
    9. Kawamoto, Carlos Tadao & Wright, James Terence Coulter & Spers, Renata Giovinazzo & de Carvalho, Daniel Estima, 2019. "Can we make use of perception of questions' easiness in Delphi-like studies? Some results from an experiment with an alternative feedback," Technological Forecasting and Social Change, Elsevier, vol. 140(C), pages 296-305.
    10. Meijering, Jurian Vincent & Tobi, Hilde, 2018. "The effects of feeding back experts’ own initial ratings in Delphi studies: A randomized trial," International Journal of Forecasting, Elsevier, vol. 34(2), pages 216-224.
    11. Frevel, Nicolas & Beiderbeck, Daniel & Schmidt, Sascha L., 2022. "The impact of technology on sports – A prospective study," Technological Forecasting and Social Change, Elsevier, vol. 182(C).
    12. C. J. Torrecilla-Salinas & O. Troyer & M. J. Escalona & M. Mejías, 2019. "A Delphi-based expert judgment method applied to the validation of a mature Agile framework for Web development projects," Information Technology and Management, Springer, vol. 20(1), pages 9-40, March.
    13. Moutinho, José da Assunção & Fernandes, Gabriela & Rabechini, Roque, 2024. "Evaluation in design science: A framework to support project studies in the context of University Research Centres," Evaluation and Program Planning, Elsevier, vol. 102(C).
    14. Chen, Kaihua & Ren, Zhipeng & Mu, Shijun & Sun, Tara Qian & Mu, Rongping, 2020. "Integrating the Delphi survey into scenario planning for China's renewable energy development strategy towards 2030," Technological Forecasting and Social Change, Elsevier, vol. 158(C).
    15. Wei Qi & Xiumei Guo & Xia Wu & Dora Marinova & Jin Fan, 2018. "Do the sunk cost effect and cognitive dissonance increase risk perception? An empirical study in the context of city smog," Quality & Quantity: International Journal of Methodology, Springer, vol. 52(5), pages 2269-2289, September.
    16. Rowe, Gene & Wright, George, 1999. "The Delphi technique as a forecasting tool: issues and analysis," International Journal of Forecasting, Elsevier, vol. 15(4), pages 353-375, October.
    17. Muhammad Raza, Normalisa Md Isa, Shamsul Huda Bt Abd Rani, 2019. "Effect of Celebrity-Endorsed Advertisement and Entrepreneurial Marketing on Purchase Behavior of Smartphone Consumers in Pakistan," Journal of Management Sciences, Geist Science, Iqra University, Faculty of Business Administration, vol. 6(1), pages 15-29, March.
    18. Mauksch, Stefanie & von der Gracht, Heiko A. & Gordon, Theodore J., 2020. "Who is an expert for foresight? A review of identification methods," Technological Forecasting and Social Change, Elsevier, vol. 154(C).
    19. Steininger, Dennis M. & Gatzemeier, Simon, 2019. "Digitally forecasting new music product success via active crowdsourcing," Technological Forecasting and Social Change, Elsevier, vol. 146(C), pages 167-180.
    20. Arturo Realyvásquez Vargas & Jorge Luis García Alcaraz & Julio Blanco Fernández, 2016. "Developing and validating a macro-ergonomic compatibility questionnaire," Contaduría y Administración, Accounting and Management, vol. 61(3), pages 478-498, Julio-Sep.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:tefoso:v:170:y:2021:i:c:s0040162521003292. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.sciencedirect.com/science/journal/00401625 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.