IDEAS home Printed from https://ideas.repec.org/a/nos/ddldem/69.html
   My bibliography  Save this article

Development of methodological support for improving the quality of expert assessment of business processes

Author

Listed:
  • Vitalii Antoshchuk

    (Odessa National Polytechnic University)

  • Volodymyr Filippov

    (Odessa National Polytechnic University)

  • Varvara Kuvaieva

    (Odessa National Polytechnic University)

Abstract

The object of research is the process of forming a collective expert assessment with increased reliability in making management decisions in business structures by an expanded team of experts. One of the most problematic places in the expert assessment of management decisions is the complexity of forming a competent expert team and the rather high cost of the expertise. In recent years, there has been a tendency for expert assessment with an expanded team of experts. In this case, not only professional experts are involved in the examination, but also all persons wishing to take part in solving the problem. In this case, the reliability of the examination raises doubts. In connection with the participation in expert assessment of persons who do not have experience in expert work, a wide range of expert assessments is possible. The analysis of the current state of the methods of expert assessment in business is carried out. It has been established that the Delphi method, which was most used until recently, does not meet modern requirements. More progressive methods are based on mathematical consensus theory. Consensus is understood as the degree of correlation of individual expert assessments performed in rank scales. In the course of the study, formalized mathematical approaches to the organization of collective expertise were used. A method for processing the results of an examination with an expanded composition of experts was developed. The developed methodology is focused on identifying experts with insufficient qualifications. The methodology allows for a step-by-step assessment of the reliability of the collective expert decision by assessing the Kendall concordance coefficient. It is shown that the phased exclusion of assessments by experts with insufficient qualifications allows increasing the level of consensus, the quality and reliability of the collective expert assessment. The developed methodology has been tested in a really functioning enterprise to make a decision on the exit strategy of the enterprise from their crisis. The use of the developed methodology has made it possible to significantly increase the reliability of the examination results, assessed by the concordance coefficient. The results are useful for practical application in business structures when conducting expert examinations involving a wide range of participants.

Suggested Citation

  • Vitalii Antoshchuk & Volodymyr Filippov & Varvara Kuvaieva, 2021. "Development of methodological support for improving the quality of expert assessment of business processes," Technology audit and production reserves, Socionet;Technology audit and production reserves, vol. 1(4(57)), pages 22-27.
    • Handle: RePEc:nos:ddldem:69
    as

    Download full text from publisher

    File URL: http://journals.uran.ua/tarp/article/view/225336
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. David V. Budescu & Eva Chen, 2015. "Identifying Expertise to Extract the Wisdom of Crowds," Management Science, INFORMS, vol. 61(2), pages 267-280, February.
    2. Cook, Wade D. & Kress, Moshe & Seiford, Lawrence M., 1997. "A general framework for distance-based consensus in ordinal ranking models," European Journal of Operational Research, Elsevier, vol. 96(2), pages 392-397, January.
    3. Kauko, Karlo & Palmroos, Peter, 2014. "The Delphi method in forecasting financial markets— An experimental study," International Journal of Forecasting, Elsevier, vol. 30(2), pages 313-327.
    4. Evangelos Triantaphyllou & Fujun Hou & Juri Yanase, 2020. "Analysis of the Final Ranking Decisions Made by Experts After a Consensus has Been Reached in Group Decision Making," Group Decision and Negotiation, Springer, vol. 29(2), pages 271-291, April.
    5. Yucheng Dong & Jiuping Xu, 2016. "Consensus Building in Group Decision Making," Springer Books, Springer, number 978-981-287-892-2, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Liu, Fang & Chen, Ya-Ru & Zhou, Da-Hai, 2023. "A two-dimensional approach to flexibility degree of XOR numbers with application to group decision making," Mathematics and Computers in Simulation (MATCOM), Elsevier, vol. 207(C), pages 267-287.
    2. Dan Zhu & Qingwei Wang & John Goddard, 2022. "A new hedging hypothesis regarding prediction interval formation in stock price forecasting," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 41(4), pages 697-717, July.
    3. Bernd Frick & Franziska Prockl, 2018. "Information Precision In Online Communities: Player Valuations On Www.Transfermarkt.De," Working Papers Dissertations 37, Paderborn University, Faculty of Business Administration and Economics.
    4. repec:cup:judgdm:v:14:y:2019:i:4:p:395-411 is not listed on IDEAS
    5. Benchimol, Jonathan & El-Shagi, Makram & Saadon, Yossi, 2022. "Do expert experience and characteristics affect inflation forecasts?," Journal of Economic Behavior & Organization, Elsevier, vol. 201(C), pages 205-226.
    6. Jaspersen, Johannes G., 2022. "Convex combinations in judgment aggregation," European Journal of Operational Research, Elsevier, vol. 299(2), pages 780-794.
    7. Brown, Alasdair & Reade, J. James, 2019. "The wisdom of amateur crowds: Evidence from an online community of sports tipsters," European Journal of Operational Research, Elsevier, vol. 272(3), pages 1073-1081.
    8. Patrick Afflerbach & Christopher Dun & Henner Gimpel & Dominik Parak & Johannes Seyfried, 2021. "A Simulation-Based Approach to Understanding the Wisdom of Crowds Phenomenon in Aggregating Expert Judgment," Business & Information Systems Engineering: The International Journal of WIRTSCHAFTSINFORMATIK, Springer;Gesellschaft für Informatik e.V. (GI), vol. 63(4), pages 329-348, August.
    9. Fujun Hou, 2015. "A Consensus Gap Indicator and Its Application to Group Decision Making," Group Decision and Negotiation, Springer, vol. 24(3), pages 415-428, May.
    10. Feliciani, Thomas & Morreau, Michael & Luo, Junwen & Lucas, Pablo & Shankar, Kalpana, 2022. "Designing grant-review panels for better funding decisions: Lessons from an empirically calibrated simulation model," Research Policy, Elsevier, vol. 51(4).
    11. Shinitzky, Hilla & Shemesh, Yhonatan & Leiser, David & Gilead, Michael, 2024. "Improving geopolitical forecasts with 100 brains and one computer," International Journal of Forecasting, Elsevier, vol. 40(3), pages 958-970.
    12. Bora Erdamar & José Luis Garcia-Lapresta & David Pérez-Roman & Remzi Sanver, 2012. "Measuring consensus in a preference-approval context," Working Papers hal-00681297, HAL.
    13. Marcellin Martinie & Tom Wilkening & Piers D L Howe, 2020. "Using meta-predictions to identify experts in the crowd when past performance is unknown," PLOS ONE, Public Library of Science, vol. 15(4), pages 1-11, April.
    14. Zhang, Hengjie & Dong, Yucheng & Chiclana, Francisco & Yu, Shui, 2019. "Consensus efficiency in group decision making: A comprehensive comparative study and its optimal design," European Journal of Operational Research, Elsevier, vol. 275(2), pages 580-598.
    15. Saul Estrin & Susanna Khavul & Mike Wright, 2022. "Soft and hard information in equity crowdfunding: network effects in the digitalization of entrepreneurial finance," Small Business Economics, Springer, vol. 58(4), pages 1761-1781, April.
    16. Wang, Xiaoqian & Hyndman, Rob J. & Li, Feng & Kang, Yanfei, 2023. "Forecast combinations: An over 50-year review," International Journal of Forecasting, Elsevier, vol. 39(4), pages 1518-1547.
    17. Anca M. Hanea & Marissa F. McBride & Mark A. Burgman & Bonnie C. Wintle, 2018. "The Value of Performance Weights and Discussion in Aggregated Expert Judgments," Risk Analysis, John Wiley & Sons, vol. 38(9), pages 1781-1794, September.
    18. Yuyu Fan & David V. Budescu & David Mandel & Mark Himmelstein, 2019. "Improving Accuracy by Coherence Weighting of Direct and Ratio Probability Judgments," Decision Analysis, INFORMS, vol. 16(3), pages 197-217, September.
    19. Kawamoto, Carlos Tadao & Wright, James Terence Coulter & Spers, Renata Giovinazzo & de Carvalho, Daniel Estima, 2019. "Can we make use of perception of questions' easiness in Delphi-like studies? Some results from an experiment with an alternative feedback," Technological Forecasting and Social Change, Elsevier, vol. 140(C), pages 296-305.
    20. Johannes Müller-Trede & Shoham Choshen-Hillel & Meir Barneron & Ilan Yaniv, 2018. "The Wisdom of Crowds in Matters of Taste," Management Science, INFORMS, vol. 64(4), pages 1779-1803, April.
    21. Mavrodiev, Pavlin & Schweitzer, Frank, 2021. "The ambigous role of social influence on the wisdom of crowds: An analytic approach," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 567(C).

    More about this item

    Keywords

    collective expert assessment; expanded team of experts; a variety of alternatives; concordance coefficient; examination reliability;
    All these keywords.

    JEL classification:

    • D70 - Microeconomics - - Analysis of Collective Decision-Making - - - General

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nos:ddldem:69. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Алина Макаренко (email available below). General contact details of provider: http://socionet.ru/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.