IDEAS home Printed from https://ideas.repec.org/a/eee/epplan/v62y2017icp56-63.html
   My bibliography  Save this article

Concept mapping internal validity: A case of misconceived mapping?

Author

Listed:
  • Péladeau, Normand
  • Dagenais, Christian
  • Ridde, Valéry

Abstract

Since the early 1990s, the concept mapping technique developed by William M. K. Trochim has been widely used by evaluators for program development and evaluation and proven to be an invaluable tool for evaluators and program planners. The technique combines qualitative and statistical analysis and is designed to help identify and prioritize the components, dimensions, and particularities of a given reality. The aim of this paper is to propose an alternative way of conducting the statistical analysis to make the technique even more useful and the results easier to interpret. We posit that some methodological choices made at the inception stage of the technique were ill informed, producing maps of participants’ points-of-view that were not optimal representations of their reality. Such a depiction resulted from the statistical analysis process by which multidimensional scaling (MDS) is being applied on the similarity matrix, followed by a hierarchical cluster analysis (HCA) on the Euclidian distances between statements as plotted on the resulting two-dimensional MDS map. As an alternative, we suggest that HCA should be performed first and MDS second, rather than the reverse. To support this proposal, we present three levels of argument: 1) a logical argument backed up by expert opinions on this issue; 2) statistical evidence of the superiority of our proposed approach and 3) the results of a social validation experiment.

Suggested Citation

  • Péladeau, Normand & Dagenais, Christian & Ridde, Valéry, 2017. "Concept mapping internal validity: A case of misconceived mapping?," Evaluation and Program Planning, Elsevier, vol. 62(C), pages 56-63.
    • Handle: RePEc:eee:epplan:v:62:y:2017:i:c:p:56-63
    DOI: 10.1016/j.evalprogplan.2017.02.005
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0149718916301446
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.evalprogplan.2017.02.005?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Céline Mercier & Myra Piat & Normand Peladeau & Christian Dagenais, 2000. "An Application of Theory-Driven Evaluation to a Drop-In Youth Center," Evaluation Review, , vol. 24(1), pages 73-91, February.
    2. Rosas, Scott R. & Camphausen, Lauren C., 2007. "The use of concept mapping for scale development and validation in evaluation," Evaluation and Program Planning, Elsevier, vol. 30(2), pages 125-135, May.
    3. Warren Torgerson, 1952. "Multidimensional scaling: I. Theory and method," Psychometrika, Springer;The Psychometric Society, vol. 17(4), pages 401-419, December.
    4. Trochim, William M. K., 1989. "An introduction to concept mapping for planning and evaluation," Evaluation and Program Planning, Elsevier, vol. 12(1), pages 1-16, January.
    5. Sutherland, Stephanie & Katz, Steven, 2005. "Concept mapping methodology: A catalyst for organizational learning," Evaluation and Program Planning, Elsevier, vol. 28(3), pages 257-269, August.
    6. Christian Dagenais & Renée Pinard & Marjolaine St-Pierre & Mélodie Briand-Lamarche & Amina Kay Cantave & Normand Péladeau, 2016. "Using concept mapping to identify conditions that foster knowledge translation from the perspective of school practitioners," Research Evaluation, Oxford University Press, vol. 25(1), pages 70-78.
    7. Trochim, William M. K., 1989. "Concept mapping : Soft science or hard art?," Evaluation and Program Planning, Elsevier, vol. 12(1), pages 87-110, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Houle, Andrée-Anne & Besnard, Thérèse & Bérubé, Annie & Dagenais, Christian, 2018. "Factors that influence parent recruitment into prevention programs in early childhood: A concept map of parents', practitioners', and administrators' points of view," Children and Youth Services Review, Elsevier, vol. 85(C), pages 127-136.
    2. Ma, Jie & Tse, Ying Kei & Wang, Xiaojun & Zhang, Minhao, 2019. "Examining customer perception and behaviour through social media research – An empirical study of the United Airlines overbooking crisis," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 127(C), pages 192-205.
    3. Loubna Belaid & Magalie Benoit & Navdeep Kaur & Azari Lili & Valery Ridde, 2020. "Population Health Intervention Implementation Among Migrants With Precarious Status in Montreal: Underlying Theory and Key Challenges," SAGE Open, , vol. 10(2), pages 21582440209, May.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jabbar, Amina M. & Abelson, Julia, 2011. "Development of a framework for effective community engagement in Ontario, Canada," Health Policy, Elsevier, vol. 101(1), pages 59-69, June.
    2. Sofia Patsali, 2019. "Opening the black box of university-suppliers' co-invention: some field study evidence," Working Papers of BETA 2019-46, Bureau d'Economie Théorique et Appliquée, UDS, Strasbourg.
    3. Van Holen, Frank & Van Loock, Julie & Belenger, Laurence & Vanderfaeillie, Johan, 2017. "Concept mapping the needs of grandmothers who take care of their grandchildren in formal foster care in Flanders," Children and Youth Services Review, Elsevier, vol. 83(C), pages 159-167.
    4. Klenk, Nicole L. & Hickey, Gordon M., 2011. "A virtual and anonymous, deliberative and analytic participation process for planning and evaluation: The Concept Mapping Policy Delphi," International Journal of Forecasting, Elsevier, vol. 27(1), pages 152-165, January.
    5. Rosas, Scott R. & Ridings, John W., 2017. "The use of concept mapping in measurement development and evaluation: Application and future directions," Evaluation and Program Planning, Elsevier, vol. 60(C), pages 265-276.
    6. Miller, J. Jay & Duron, Jacquelynn F. & Donohue-Dioh, Jessica & Geiger, Jennifer M., 2018. "Conceptualizing effective legal representation for Foster youth: A group concept mapping study," Children and Youth Services Review, Elsevier, vol. 91(C), pages 271-278.
    7. Johnsen, Jeffrey A. & Biegel, David E. & Shafran, Robert, 2000. "Concept mapping in mental health: uses and adaptations," Evaluation and Program Planning, Elsevier, vol. 23(1), pages 67-75, February.
    8. Kaplan, Deanna M. & deBlois, Madeleine & Dominguez, Violeta & Walsh, Michele E., 2016. "Studying the teaching of kindness: A conceptual model for evaluating kindness education programs in schools," Evaluation and Program Planning, Elsevier, vol. 58(C), pages 160-170.
    9. Brown, Jason D. & Sigvaldason, Nadine & Bednar, Lisa M., 2005. "Foster parent perceptions of placement needs for children with a fetal alcohol spectrum disorder," Children and Youth Services Review, Elsevier, vol. 27(3), pages 309-327, March.
    10. Klenk, Nicole L. & Hickey, Gordon M., 2011. "A virtual and anonymous, deliberative and analytic participation process for planning and evaluation: The Concept Mapping Policy Delphi," International Journal of Forecasting, Elsevier, vol. 27(1), pages 152-165.
    11. Maier, Konradin & Stix, Volker, 2013. "A semi-automated approach for structuring multi criteria decision problems," European Journal of Operational Research, Elsevier, vol. 225(3), pages 487-496.
    12. C. Pons-Morera & L. Canós-Darós & I. Gil-Pechuan, 2018. "A model of collaborative innovation between local government and tourism operators," Service Business, Springer;Pan-Pacific Business Association, vol. 12(1), pages 143-168, March.
    13. Jay Miller, J. & Benner, Kalea & Kheibari, Athena & Washington, Earl, 2017. "Conceptualizing on-campus support programs for collegiate foster youth and alumni: A plan for action," Children and Youth Services Review, Elsevier, vol. 83(C), pages 57-67.
    14. Dare, Lynn & Nowicki, Elizabeth, 2019. "Engaging children and youth in research and evaluation using group concept mapping," Evaluation and Program Planning, Elsevier, vol. 76(C), pages 1-1.
    15. Michalski, Greg V. & Cousins, J. Bradley, 2000. "Differences in stakeholder perceptions about training evaluation: a concept mapping/pattern matching investigation," Evaluation and Program Planning, Elsevier, vol. 23(2), pages 211-230, May.
    16. Trochim, William M., 2017. "Hindsight is 20/20: Reflections on the evolution of concept mapping," Evaluation and Program Planning, Elsevier, vol. 60(C), pages 176-185.
    17. Rosas, Scott R. & Kane, Mary, 2012. "Quality and rigor of the concept mapping methodology: A pooled study analysis," Evaluation and Program Planning, Elsevier, vol. 35(2), pages 236-245.
    18. McLinden, Daniel, 2017. "And then the internet happened: Thoughts on the future of concept mapping," Evaluation and Program Planning, Elsevier, vol. 60(C), pages 293-300.
    19. Schalock, Robert L. & Lee, Tim & Verdugo, Miguel & Swart, Kees & Claes, Claudia & van Loon, Jos & Lee, Chun-Shin, 2014. "An evidence-based approach to organization evaluation and change in human service organizations evaluation and program planning," Evaluation and Program Planning, Elsevier, vol. 45(C), pages 110-118.
    20. Donnelly, James P., 2017. "A systematic review of concept mapping dissertations," Evaluation and Program Planning, Elsevier, vol. 60(C), pages 186-193.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:epplan:v:62:y:2017:i:c:p:56-63. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/evalprogplan .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.