IDEAS home Printed from https://ideas.repec.org/a/eee/ejores/v305y2023i1p475-492.html
   My bibliography  Save this article

Collaborative development of composite indices from qualitative value judgements: The EURO-HEALTHY Population Health Index model

Author

Listed:
  • Bana e Costa, Carlos A.
  • Oliveira, Mónica D.
  • Vieira, Ana C.L.
  • Freitas, Liliana
  • Rodrigues, Teresa C.
  • Bana e Costa, João
  • Freitas, Ângela
  • Santana, Paula

Abstract

Promoting participation and combining evidence with expert knowledge in constructing composite indices that can be meaningfully interpreted are key open challenges identified in performance measurement literature. To address them, this article proposes a sociotechnical decision-aid process, under a collaborative value-modelling framework, rooted in both process consultation and multicriteria value measurement principles. It consists of two phases (structuring and model building), each one developed in a sequence of non-face-to-face (Web-Delphi) and face-to-face (decision conferencing) group processes, designed to allow for increased participation. Performance indicators are selected and structured in a value-tree (phase one), upon which a group hierarchical additive value-function model is built (phase two), supported by the MACBETH multicriteria decision-support system. Collaborative questioning protocols allow to elicit and reconcile individual qualitative value judgements, in a Web-Delphi platform. The collective knowledge thus acquired is used in decision conferencing with a core group to define value functions and weights. Extensive sensitivity and robustness analyses serve to test the requisiteness of the model, giving rise to a shared structure of composite indices. The proposed approach was shaped within the EURO-HEALTHY project, to create a Population Health Index intended to be a tool enabling policymakers to analyse population health and related inequalities at the regional level across the European Union, and design improvement policies. Many stakeholders and experts from different intervention and knowledge areas, and located across Europe, participated in the different modelling activities, as detailed in this article. Research paths derived from learnings acquired are suggested to further improve process effectiveness and flexibility.

Suggested Citation

  • Bana e Costa, Carlos A. & Oliveira, Mónica D. & Vieira, Ana C.L. & Freitas, Liliana & Rodrigues, Teresa C. & Bana e Costa, João & Freitas, Ângela & Santana, Paula, 2023. "Collaborative development of composite indices from qualitative value judgements: The EURO-HEALTHY Population Health Index model," European Journal of Operational Research, Elsevier, vol. 305(1), pages 475-492.
    • Handle: RePEc:eee:ejores:v:305:y:2023:i:1:p:475-492
    DOI: 10.1016/j.ejor.2022.05.037
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S037722172200426X
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ejor.2022.05.037?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Rodrigues, Teresa C. & Montibeller, Gilberto & Oliveira, Mónica D. & Bana e Costa, Carlos A., 2017. "Modelling multicriteria value interactions with Reasoning Maps," European Journal of Operational Research, Elsevier, vol. 258(3), pages 1054-1071.
    2. Wątróbski, Jarosław & Jankowski, Jarosław & Ziemba, Paweł & Karczmarczyk, Artur & Zioło, Magdalena, 2019. "Generalised framework for multi-criteria method selection," Omega, Elsevier, vol. 86(C), pages 107-124.
    3. P.C. Smith, 2002. "Measuring health system performance," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 3(3), pages 145-148, September.
    4. Bana e Costa, Carlos A. & Ensslin, Leonardo & Correa, Emerson C. & Vansnick, Jean-Claude, 1999. "Decision Support Systems in action: Integrated application in a multicriteria decision aid process," European Journal of Operational Research, Elsevier, vol. 113(2), pages 315-335, March.
    5. Douwe Postmus & Tommi Tervonen & Gert Valkenhoef & Hans Hillege & Erik Buskens, 2014. "A multi-criteria decision analysis perspective on the health economic evaluation of medical interventions," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 15(7), pages 709-716, September.
    6. Paula Santana & Ângela Freitas & Cláudia Costa & Iwa Stefanik & Gonçalo Santinha & Thomas Krafft & Eva Pilot, 2020. "The Role of Cohesion Policy Funds in Decreasing the Health Gaps Measured by the EURO-HEALTHY Population Health Index," IJERPH, MDPI, vol. 17(5), pages 1-21, February.
    7. Bana e Costa, Carlos A. & Carnero, María Carmen & Oliveira, Mónica Duarte, 2012. "A multi-criteria model for auditing a Predictive Maintenance Programme," European Journal of Operational Research, Elsevier, vol. 217(2), pages 381-393.
    8. Mónica D. Oliveira & Inês Mataloto & Panos Kanavos, 2019. "Multi-criteria decision analysis for health technology assessment: addressing methodological challenges to improve the state of the art," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 20(6), pages 891-918, August.
    9. Bouyssou, Denis, 1986. "Some remarks on the notion of compensation in MCDM," European Journal of Operational Research, Elsevier, vol. 26(1), pages 150-160, July.
    10. Dennis G. Fryback & Ralph L. Keeney, 1983. "Constructing a Complex Judgmental Model: An Index of Trauma Severity," Management Science, INFORMS, vol. 29(8), pages 869-883, August.
    11. Gregory, Amanda J. & Atkins, Jonathan P. & Midgley, Gerald & Hodgson, Anthony M., 2020. "Stakeholder identification and engagement in problem structuring interventions," European Journal of Operational Research, Elsevier, vol. 283(1), pages 321-340.
    12. Kaltenthaler, Eva & Maheswaran, Ravi & Beverley, Catherine, 2004. "Population-based health indexes: a systematic review," Health Policy, Elsevier, vol. 68(2), pages 245-255, May.
    13. Jay Simon & Craig W. Kirkwood & L. Robin Keller, 2014. "Decision Analysis with Geographically Varying Outcomes: Preference Models and Illustrative Applications," Operations Research, INFORMS, vol. 62(1), pages 182-194, February.
    14. Vieira, Ana C.L. & Oliveira, Mónica D. & Bana e Costa, Carlos A., 2020. "Enhancing knowledge construction processes within multicriteria decision analysis: The Collaborative Value Modelling framework," Omega, Elsevier, vol. 94(C).
    15. Franco, L. Alberto & Lord, Ewan, 2011. "Understanding multi-methodology: Evaluating the perceived impact of mixing methods for group budgetary decisions," Omega, Elsevier, vol. 39(3), pages 362-372, June.
    16. Mónica D. Oliveira & Teresa C. Rodrigues & Carlos A. Bana e Costa & Armando Brito de Sá, 2012. "Prioritizing health care interventions: A multicriteria resource allocation model to inform the choice of community care programmes," International Series in Operations Research & Management Science, in: Elena Tànfani & Angela Testi (ed.), Advanced Decision Making Methods Applied to Health Care, chapter 0, pages 141-154, Springer.
    17. Bana e Costa, Carlos A. & Corrêa, Émerson C. & De Corte, Jean-Marie & Vansnick, Jean-Claude, 2002. "Facilitating bid evaluation in public call for tenders: a socio-technical approach," Omega, Elsevier, vol. 30(3), pages 227-242, June.
    18. Kindig, D.A. & Stoddart, G., 2003. "What is population health?," American Journal of Public Health, American Public Health Association, vol. 93(3), pages 380-383.
    19. Dyer, JS & Lorber, HW, 1982. "The multiattribute evaluation of program-planning contractors," Omega, Elsevier, vol. 10(6), pages 673-678.
    20. James S. Dyer & Rakesh K. Sarin, 1979. "Measurable Multiattribute Value Functions," Operations Research, INFORMS, vol. 27(4), pages 810-822, August.
    21. Scholten, Lisa & Schuwirth, Nele & Reichert, Peter & Lienert, Judit, 2015. "Tackling uncertainty in multi-criteria decision analysis – An application to water supply infrastructure planning," European Journal of Operational Research, Elsevier, vol. 242(1), pages 243-260.
    22. P. Thokala & A. Duenas, 2012. "Multiple Criteria Decision Analysis for Health Technology Assessment," Post-Print hal-00800398, HAL.
    23. Franco, L. Alberto & Montibeller, Gilberto, 2010. "Facilitated modelling in operational research," European Journal of Operational Research, Elsevier, vol. 205(3), pages 489-500, September.
    24. Murat Köksalan & Jyrki Wallenius & Stanley Zionts, 2011. "Multiple Criteria Decision Making:From Early History to the 21st Century," World Scientific Books, World Scientific Publishing Co. Pte. Ltd., number 8042, October.
    25. Chih-Ching Yang, 2017. "Measuring health indicators and allocating health resources: a DEA-based approach," Health Care Management Science, Springer, vol. 20(3), pages 365-378, September.
    26. Lienert, Judit & Duygan, Mert & Zheng, Jun, 2016. "Preference stability over time with multiple elicitation methods to support wastewater infrastructure decision-making," European Journal of Operational Research, Elsevier, vol. 253(3), pages 746-760.
    27. Bottero, M. & Ferretti, V. & Figueira, J.R. & Greco, S. & Roy, B., 2018. "On the Choquet multiple criteria preference aggregation model: Theoretical and practical insights from a real-world application," European Journal of Operational Research, Elsevier, vol. 271(1), pages 120-140.
    28. Salvatore Greco & Alessio Ishizaka & Menelaos Tasiou & Gianpiero Torrisi, 2019. "On the Methodological Framework of Composite Indices: A Review of the Issues of Weighting, Aggregation, and Robustness," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 141(1), pages 61-94, January.
    29. Keeney, Ralph L., 1996. "Value-focused thinking: Identifying decision opportunities and creating alternatives," European Journal of Operational Research, Elsevier, vol. 92(3), pages 537-549, August.
    30. Ramiro Sanchez-Lopez & Carlos Bana e Costa & Bernard Baets, 2012. "The MACBETH approach for multi-criteria evaluation of development projects on cross-cutting issues," Annals of Operations Research, Springer, vol. 199(1), pages 393-408, October.
    31. Bana e Costa, Carlos A. & Oliveira, Rui Carvalho, 2002. "Assigning priorities for maintenance, repair and refurbishment in managing a municipal housing stock," European Journal of Operational Research, Elsevier, vol. 138(2), pages 380-391, April.
    32. Rowe, Gene & Wright, George, 1999. "The Delphi technique as a forecasting tool: issues and analysis," International Journal of Forecasting, Elsevier, vol. 15(4), pages 353-375, October.
    33. Nuno F. da Cruz & Rui Cunha Marques, 2017. "Structuring composite local governance indicators," Policy Studies, Taylor & Francis Journals, vol. 38(2), pages 109-129, March.
    34. Carlos A. Bana e Costa & João Carlos Lourenço & Mónica Duarte Oliveira & João C. Bana e Costa, 2014. "A Socio-technical Approach for Group Decision Support in Public Strategic Planning: The Pernambuco PPA Case," Group Decision and Negotiation, Springer, vol. 23(1), pages 5-29, January.
    35. Corner, James L., 1994. "Operationalizing approximate multiattribute utility functions for use in practice," European Journal of Operational Research, Elsevier, vol. 79(1), pages 73-84, November.
    36. Mingers, John, 2015. "Helping business schools engage with real problems: The contribution of critical realism and systems thinking," European Journal of Operational Research, Elsevier, vol. 242(1), pages 316-331.
    37. Carlos A. Bana e Costa & Jean-Marie De Corte & Jean-Claude Vansnick, 2016. "On the Mathematical Foundations of MACBETH," International Series in Operations Research & Management Science, in: Salvatore Greco & Matthias Ehrgott & José Rui Figueira (ed.), Multiple Criteria Decision Analysis, edition 2, chapter 0, pages 421-463, Springer.
    38. Bana e Costa, Carlos A. & Oliveira, Mónica D., 2012. "A multicriteria decision analysis model for faculty evaluation," Omega, Elsevier, vol. 40(4), pages 424-436.
    39. Axel C. Mühlbacher & Anika Kaczynski, 2016. "Making Good Decisions in Healthcare with Multi-Criteria Decision Analysis: The Use, Current Research and Future Development of MCDA," Applied Health Economics and Health Policy, Springer, vol. 14(1), pages 29-40, February.
    40. Fasolo, Barbara & Bana e Costa, Carlos A., 2014. "Tailoring value elicitation to decision makers' numeracy and fluency: Expressing value judgments in numbers or words," Omega, Elsevier, vol. 44(C), pages 83-90.
    41. Rouwette, Etiënne & van Kranenburg, Hans & Freeman, Edward, 2017. "Reviewing the role of stakeholders in Operational Research: A stakeholder theory perspectiveAuthor-Name: de Gooyert, Vincent," European Journal of Operational Research, Elsevier, vol. 262(2), pages 402-410.
    42. Michael Freudenberg, 2003. "Composite Indicators of Country Performance: A Critical Assessment," OECD Science, Technology and Industry Working Papers 2003/16, OECD Publishing.
    43. Gerardine DeSanctis & R. Brent Gallupe, 1987. "A Foundation for the Study of Group Decision Support Systems," Management Science, INFORMS, vol. 33(5), pages 589-609, May.
    44. Craig W. Kirkwood & Rakesh K. Sarin, 1980. "Preference Conditions for Multiattribute Value Functions," Operations Research, INFORMS, vol. 28(1), pages 225-232, February.
    45. Frederik Booysen, 2002. "An Overview and Evaluation of Composite Indices of Development," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 59(2), pages 115-151, August.
    46. O'Keefe, Robert M., 2016. "Experimental behavioural research in operational research: What we know and what we might come to know," European Journal of Operational Research, Elsevier, vol. 249(3), pages 899-907.
    47. Lawrence Phillips & Carlos Bana e Costa, 2007. "Transparent prioritisation, budgeting and resource allocation with multi-criteria decision analysis and decision conferencing," Annals of Operations Research, Springer, vol. 154(1), pages 51-68, October.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Rodrigues, Teresa C. & Montibeller, Gilberto & Oliveira, Mónica D. & Bana e Costa, Carlos A., 2017. "Modelling multicriteria value interactions with Reasoning Maps," European Journal of Operational Research, Elsevier, vol. 258(3), pages 1054-1071.
    2. Mónica D. Oliveira & Inês Mataloto & Panos Kanavos, 2019. "Multi-criteria decision analysis for health technology assessment: addressing methodological challenges to improve the state of the art," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 20(6), pages 891-918, August.
    3. Angelis, A. & Linch, M. & Montibeller, G. & Molina-Lopez, T. & Zawada, A. & Orzel, K. & Arickx, F. & Espin, J. & Kanavos, P., 2020. "Multiple Criteria Decision Analysis for HTA across four EU Member States: Piloting the Advance Value Framework," Social Science & Medicine, Elsevier, vol. 246(C).
    4. Vieira, Ana C.L. & Oliveira, Mónica D. & Bana e Costa, Carlos A., 2020. "Enhancing knowledge construction processes within multicriteria decision analysis: The Collaborative Value Modelling framework," Omega, Elsevier, vol. 94(C).
    5. Bana e Costa, Carlos A. & Oliveira, Mónica D. & Rodrigues, Teresa C. & Vieira, Ana C.L., 2023. "Desirability–doability group judgment framework for the collaborative multicriteria evaluation of public policies," LSE Research Online Documents on Economics 118192, London School of Economics and Political Science, LSE Library.
    6. Marttunen, Mika & Haara, Arto & Hjerppe, Turo & Kurttila, Mikko & Liesiö, Juuso & Mustajoki, Jyri & Saarikoski, Heli & Tolvanen, Anne, 2023. "Parallel and comparative use of three multicriteria decision support methods in an environmental portfolio problem," European Journal of Operational Research, Elsevier, vol. 307(2), pages 842-859.
    7. Fernando A. F. Ferreira & Ieva Meidutė-Kavaliauskienė & Edmundas K. Zavadskas & Marjan S. Jalali & Sandra M. J. Catarino, 2019. "A Judgment-Based Risk Assessment Framework for Consumer Loans," International Journal of Information Technology & Decision Making (IJITDM), World Scientific Publishing Co. Pte. Ltd., vol. 18(01), pages 7-33, January.
    8. Fernando A. F. Ferreira & Sérgio P. Santos, 2021. "Two decades on the MACBETH approach: a bibliometric analysis," Annals of Operations Research, Springer, vol. 296(1), pages 901-925, January.
    9. Alessio Ishizaka & Maynard Gordon, 2017. "MACBETHSort: a multiple criteria decision aid procedure for sorting strategic products," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 68(1), pages 53-61, January.
    10. Jaime Burgos & María Carmen Carnero, 2020. "Assessment of Social Responsibility in Education in Secondary Schools," Sustainability, MDPI, vol. 12(12), pages 1-38, June.
    11. Francis Marleau Donais & Irène Abi-Zeid & E. Owen D. Waygood & Roxane Lavoie, 2021. "A Framework for Post-Project Evaluation of Multicriteria Decision Aiding Processes from the Stakeholders’ Perspective: Design and Application," Group Decision and Negotiation, Springer, vol. 30(5), pages 1161-1191, October.
    12. Aris Angelis & Mark Thursz & Vlad Ratziu & Alastair O’Brien & Lawrence Serfaty & Ali Canbay & Ingolf Schiefke & Joao Bana e Costa & Pascal Lecomte & Panos Kanavos, 2020. "Early Health Technology Assessment during Nonalcoholic Steatohepatitis Drug Development: A Two-Round, Cross-Country, Multicriteria Decision Analysis," Medical Decision Making, , vol. 40(6), pages 830-845, August.
    13. Bana e Costa, Carlos A. & Oliveira, Mónica D., 2012. "A multicriteria decision analysis model for faculty evaluation," Omega, Elsevier, vol. 40(4), pages 424-436.
    14. Angelis, Aris & Kanavos, Panos, 2017. "Multiple Criteria Decision Analysis (MCDA) for evaluating new medicines in Health Technology Assessment and beyond: The Advance Value Framework," Social Science & Medicine, Elsevier, vol. 188(C), pages 137-156.
    15. Carlos A. Bana e Costa & João C. Lourenço & Manuel P. Chagas & João C. Bana e Costa, 2008. "Development of Reusable Bid Evaluation Models for the Portuguese Electric Transmission Company," Decision Analysis, INFORMS, vol. 5(1), pages 22-42, March.
    16. Franco, L. Alberto & Rouwette, Etiënne A.J.A. & Korzilius, Hubert, 2016. "Different paths to consensus? The impact of need for closure on model-supported group conflict management," European Journal of Operational Research, Elsevier, vol. 249(3), pages 878-889.
    17. Franco, L. Alberto & Hämäläinen, Raimo P. & Rouwette, Etiënne A.J.A. & Leppänen, Ilkka, 2021. "Taking stock of behavioural OR: A review of behavioural studies with an intervention focus," European Journal of Operational Research, Elsevier, vol. 293(2), pages 401-418.
    18. Bana e Costa, Carlos A. & Lourenço, João C. & Chagas, Manuel P. & Bana e Costa, João C., 2007. "Development of reusable bid evaluation models for the Portugese Electric Transmission Company," LSE Research Online Documents on Economics 22697, London School of Economics and Political Science, LSE Library.
    19. Carnero, María Carmen & Gómez, Andrés, 2017. "Maintenance strategy selection in electric power distribution systems," Energy, Elsevier, vol. 129(C), pages 255-272.
    20. Fasolo, Barbara & Bana e Costa, Carlos A., 2014. "Tailoring value elicitation to decision makers' numeracy and fluency: Expressing value judgments in numbers or words," Omega, Elsevier, vol. 44(C), pages 83-90.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ejores:v:305:y:2023:i:1:p:475-492. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/eor .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.