IDEAS home Printed from https://ideas.repec.org/p/osf/osfxxx/4ufzg.html
   My bibliography  Save this paper

The Simulated Group Response Paradigm: A new approach to the study of opinion change in Delphi and other structured-group techniques

Author

Listed:
  • Bolger, Fergus
  • Rowe, Gene
  • Belton, Ian
  • Crawford, Megan M

    (University of Strathclyde)

  • Hamlin, Iain
  • Sissons, Aileen
  • Taylor Browne Lūka, Courtney
  • Vasilichi, Alexandrina
  • Wright, George

Abstract

Groups provide several benefits over individuals for judgment and decision making, but they suffer from problems too. Structured-group techniques, like Delphi, use strictly controlled information exchange between individuals to retain positive aspects of group interaction, while ameliorating negative. These methods regularly use ‘nominal’ groups that interact in a remote, distributed, and often anonymous manner, thus lending themselves to internet applications, with a consequent recent increase in popularity. However, evidence for the utility of the techniques is scant, major reasons for which being difficulties maintaining experimental control and logistical problems in recruiting sufficient empirical ‘groups’ to produce statistically meaningful results. As a solution, we present the Simulated Group Response Paradigm, where individual responses are first elicited in a pre-study – or created by the experimenter – then subsequently fed back to highly-controlled simulated groups. This paradigm facilitates investigation of factors leading to virtuous opinion change in groups, and subsequent development of structured-group techniques.

Suggested Citation

  • Bolger, Fergus & Rowe, Gene & Belton, Ian & Crawford, Megan M & Hamlin, Iain & Sissons, Aileen & Taylor Browne Lūka, Courtney & Vasilichi, Alexandrina & Wright, George, 2020. "The Simulated Group Response Paradigm: A new approach to the study of opinion change in Delphi and other structured-group techniques," OSF Preprints 4ufzg, Center for Open Science.
    • Handle: RePEc:osf:osfxxx:4ufzg
    DOI: 10.31219/osf.io/4ufzg
    as

    Download full text from publisher

    File URL: https://osf.io/download/5ef771f39fceff00b2831721/
    Download Restriction: no
    File URL: https://libkey.io/10.31219/osf.io/4ufzg?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Bolger, Fergus & Wright, George, 2017. "Use of expert knowledge to anticipate the future: Issues, analysis and directions," International Journal of Forecasting, Elsevier, vol. 33(1), pages 230-243.
    2. Graefe, Andreas & Armstrong, J. Scott, 2011. "Comparing face-to-face meetings, nominal groups, Delphi and prediction markets on an estimation task," International Journal of Forecasting, Elsevier, vol. 27(1), pages 183-195, January.
    3. Yaniv, Ilan & Kleinberger, Eli, 2000. "Advice Taking in Decision Making: Egocentric Discounting and Reputation Formation," Organizational Behavior and Human Decision Processes, Elsevier, vol. 83(2), pages 260-281, November.
    4. Dominic D. P. Johnson & James H. Fowler, 2011. "The evolution of overconfidence," Nature, Nature, vol. 477(7364), pages 317-320, September.
    5. Meijering, J.V. & Kampen, J.K. & Tobi, H., 2013. "Quantifying the development of agreement among experts in Delphi studies," Technological Forecasting and Social Change, Elsevier, vol. 80(8), pages 1607-1614.
    6. Swaab, Roderick I. & Phillips, Katherine W. & Schaerer, Michael, 2016. "Secret conversation opportunities facilitate minority influence in virtual groups: The influence on majority power, information processing, and decision quality," Organizational Behavior and Human Decision Processes, Elsevier, vol. 133(C), pages 17-32.
    7. Fergus Bolger, 2018. "The Selection of Experts for (Probabilistic) Expert Knowledge Elicitation," International Series in Operations Research & Management Science, in: Luis C. Dias & Alec Morton & John Quigley (ed.), Elicitation, chapter 0, pages 393-443, Springer.
    8. David M. Boje & J. Keith Murnighan, 1982. "Group Confidence Pressures in Iterative Decisions," Management Science, INFORMS, vol. 28(10), pages 1187-1196, October.
    9. Rowe, Gene & Wright, George, 1999. "The Delphi technique as a forecasting tool: issues and analysis," International Journal of Forecasting, Elsevier, vol. 15(4), pages 353-375, October.
    10. Rowe, Gene & Wright, George, 1996. "The impact of task characteristics on the performance of structured group forecasting techniques," International Journal of Forecasting, Elsevier, vol. 12(1), pages 73-89, March.
    11. Laku Chidambaram & Lai Lai Tung, 2005. "Is Out of Sight, Out of Mind? An Empirical Study of Social Loafing in Technology-Supported Groups," Information Systems Research, INFORMS, vol. 16(2), pages 149-168, June.
    12. Mark A Burgman & Marissa McBride & Raquel Ashton & Andrew Speirs-Bridge & Louisa Flander & Bonnie Wintle & Fiona Fidler & Libby Rumpff & Charles Twardy, 2011. "Expert Status and Performance," PLOS ONE, Public Library of Science, vol. 6(7), pages 1-7, July.
    13. Alvarado-Valencia, Jorge & Barrero, Lope H. & Önkal, Dilek & Dennerlein, Jack T., 2017. "Expertise, credibility of system forecasts and integration methods in judgmental demand forecasting," International Journal of Forecasting, Elsevier, vol. 33(1), pages 298-313.
    14. Norman Dalkey & Olaf Helmer, 1963. "An Experimental Application of the DELPHI Method to the Use of Experts," Management Science, INFORMS, vol. 9(3), pages 458-467, April.
    15. Sah, Sunita & Moore, Don A. & MacCoun, Robert J., 2013. "Cheap talk and credibility: The consequences of confidence and accuracy on advisor credibility and persuasiveness," Organizational Behavior and Human Decision Processes, Elsevier, vol. 121(2), pages 246-255.
    16. Harvey, Nigel & Fischer, Ilan, 1997. "Taking Advice: Accepting Help, Improving Judgment, and Sharing Responsibility," Organizational Behavior and Human Decision Processes, Elsevier, vol. 70(2), pages 117-133, May.
    17. Bonaccio, Silvia & Dalal, Reeshad S., 2006. "Advice taking and decision-making: An integrative literature review, and implications for the organizational sciences," Organizational Behavior and Human Decision Processes, Elsevier, vol. 101(2), pages 127-151, November.
    18. Stanislav Birko & Edward S Dove & Vural Özdemir, 2015. "Evaluation of Nine Consensus Indices in Delphi Foresight Research and Their Dependency on Delphi Survey Characteristics: A Simulation Study and Debate on Delphi Design and Interpretation," PLOS ONE, Public Library of Science, vol. 10(8), pages 1-14, August.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Belton, Ian & Wright, George & Sissons, Aileen & Bolger, Fergus & Crawford, Megan M. & Hamlin, Iain & Taylor Browne Lūka, Courtney & Vasilichi, Alexandrina, 2021. "Delphi with feedback of rationales: How large can a Delphi group be such that participants are not overloaded, de-motivated, or disengaged?," Technological Forecasting and Social Change, Elsevier, vol. 170(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mauksch, Stefanie & von der Gracht, Heiko A. & Gordon, Theodore J., 2020. "Who is an expert for foresight? A review of identification methods," Technological Forecasting and Social Change, Elsevier, vol. 154(C).
    2. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    3. Winkler, Jens & Moser, Roger, 2016. "Biases in future-oriented Delphi studies: A cognitive perspective," Technological Forecasting and Social Change, Elsevier, vol. 105(C), pages 63-76.
    4. Meijering, Jurian V. & Tobi, Hilde, 2016. "The effect of controlled opinion feedback on Delphi features: Mixed messages from a real-world Delphi experiment," Technological Forecasting and Social Change, Elsevier, vol. 103(C), pages 166-173.
    5. Meijering, Jurian Vincent & Tobi, Hilde, 2018. "The effects of feeding back experts’ own initial ratings in Delphi studies: A randomized trial," International Journal of Forecasting, Elsevier, vol. 34(2), pages 216-224.
    6. Förster, Bernadette & von der Gracht, Heiko, 2014. "Assessing Delphi panel composition for strategic foresight — A comparison of panels based on company-internal and external participants," Technological Forecasting and Social Change, Elsevier, vol. 84(C), pages 215-229.
    7. Kawamoto, Carlos Tadao & Wright, James Terence Coulter & Spers, Renata Giovinazzo & de Carvalho, Daniel Estima, 2019. "Can we make use of perception of questions' easiness in Delphi-like studies? Some results from an experiment with an alternative feedback," Technological Forecasting and Social Change, Elsevier, vol. 140(C), pages 296-305.
    8. Barrios, Maite & Guilera, Georgina & Nuño, Laura & Gómez-Benito, Juana, 2021. "Consensus in the delphi method: What makes a decision change?," Technological Forecasting and Social Change, Elsevier, vol. 163(C).
    9. Van Swol, Lyn M., 2011. "Forecasting another's enjoyment versus giving the right answer: Trust, shared values, task effects, and confidence in improving the acceptance of advice," International Journal of Forecasting, Elsevier, vol. 27(1), pages 103-120, January.
    10. Van Swol, Lyn M., 2011. "Forecasting another’s enjoyment versus giving the right answer: Trust, shared values, task effects, and confidence in improving the acceptance of advice," International Journal of Forecasting, Elsevier, vol. 27(1), pages 103-120.
    11. Jack B. Soll & Asa B. Palley & Christina A. Rader, 2022. "The Bad Thing About Good Advice: Understanding When and How Advice Exacerbates Overconfidence," Management Science, INFORMS, vol. 68(4), pages 2949-2969, April.
    12. Bonaccorsi, Andrea & Apreda, Riccardo & Fantoni, Gualtiero, 2020. "Expert biases in technology foresight. Why they are a problem and how to mitigate them," Technological Forecasting and Social Change, Elsevier, vol. 151(C).
    13. Kopyto, Matthias & Lechler, Sabrina & von der Gracht, Heiko A. & Hartmann, Evi, 2020. "Potentials of blockchain technology in supply chain management: Long-term judgments of an international expert panel," Technological Forecasting and Social Change, Elsevier, vol. 161(C).
    14. Blunden, Hayley & Logg, Jennifer M. & Brooks, Alison Wood & John, Leslie K. & Gino, Francesca, 2019. "Seeker beware: The interpersonal costs of ignoring advice," Organizational Behavior and Human Decision Processes, Elsevier, vol. 150(C), pages 83-100.
    15. Nur, Suardi & Burton, Bruce & Bergmann, Ariel, 2023. "Evidence on optimal risk allocation models for Indonesian geothermal projects under PPP contracts," Utilities Policy, Elsevier, vol. 81(C).
    16. Philipp Ecken & Richard Pibernik, 2016. "Hit or Miss: What Leads Experts to Take Advice for Long-Term Judgments?," Management Science, INFORMS, vol. 62(7), pages 2002-2021, July.
    17. Önkal, Dilek & Sinan Gönül, M. & Goodwin, Paul & Thomson, Mary & Öz, Esra, 2017. "Evaluating expert advice in forecasting: Users’ reactions to presumed vs. experienced credibility," International Journal of Forecasting, Elsevier, vol. 33(1), pages 280-297.
    18. Gehrig, Thomas & Güth, Werner & Leví0nský, René & Popova, Vera, 2010. "On the evolution of professional consulting," Journal of Economic Behavior & Organization, Elsevier, vol. 76(1), pages 113-126, October.
    19. Litsiou, Konstantia & Polychronakis, Yiannis & Karami, Azhdar & Nikolopoulos, Konstantinos, 2022. "Relative performance of judgmental methods for forecasting the success of megaprojects," International Journal of Forecasting, Elsevier, vol. 38(3), pages 1185-1196.
    20. Logg, Jennifer M. & Minson, Julia A. & Moore, Don A., 2019. "Algorithm appreciation: People prefer algorithmic to human judgment," Organizational Behavior and Human Decision Processes, Elsevier, vol. 151(C), pages 90-103.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:osf:osfxxx:4ufzg. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: OSF (email available below). General contact details of provider: https://osf.io/preprints/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.