IDEAS home Printed from https://ideas.repec.org/a/eee/epplan/v35y2012i2p236-245.html
   My bibliography  Save this article

Quality and rigor of the concept mapping methodology: A pooled study analysis

Author

Listed:
  • Rosas, Scott R.
  • Kane, Mary

Abstract

The use of concept mapping in research and evaluation has expanded dramatically over the past 20 years. Researchers in academic, organizational, and community-based settings have applied concept mapping successfully without the benefit of systematic analyses across studies to identify the features of a methodologically sound study. Quantitative characteristics and estimates of quality and rigor that may guide for future studies are lacking. To address this gap, we conducted a pooled analysis of 69 concept mapping studies to describe characteristics across study phases, generate specific indicators of validity and reliability, and examine the relationship between select study characteristics and quality indicators. Individual study characteristics and estimates were pooled and quantitatively summarized, describing the distribution, variation and parameters for each. In addition, variation in the concept mapping data collection in relation to characteristics and estimates was examined. Overall, results suggest concept mapping yields strong internal representational validity and very strong sorting and rating reliability estimates. Validity and reliability were consistently high despite variation in participation and task completion percentages across data collection modes. The implications of these findings as a practical reference to assess the quality and rigor for future concept mapping studies are discussed.

Suggested Citation

  • Rosas, Scott R. & Kane, Mary, 2012. "Quality and rigor of the concept mapping methodology: A pooled study analysis," Evaluation and Program Planning, Elsevier, vol. 35(2), pages 236-245.
    • Handle: RePEc:eee:epplan:v:35:y:2012:i:2:p:236-245
    DOI: 10.1016/j.evalprogplan.2011.10.003
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0149718911000784
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.evalprogplan.2011.10.003?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Joanna Sale & Kevin Brazil, 2004. "A Strategy to Identify Critical Appraisal Criteria for Primary Mixed-Method Studies," Quality & Quantity: International Journal of Methodology, Springer, vol. 38(4), pages 351-365, August.
    2. Dumont, Jeanne, 1989. "Validity of multidimensional scaling in the context of structured conceptualization," Evaluation and Program Planning, Elsevier, vol. 12(1), pages 81-86, January.
    3. J. Kruskal, 1964. "Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis," Psychometrika, Springer;The Psychometric Society, vol. 29(1), pages 1-27, March.
    4. Keith, Douglas, 1989. "Refining concept maps: Methodological issues and an example," Evaluation and Program Planning, Elsevier, vol. 12(1), pages 75-80, January.
    5. Trochim, W.M. & Cabrera, D.A. & Milstein, B. & Gallagher, R.S. & Leischow, S.J., 2006. "Practical challenges of systems thinking and modeling in public health," American Journal of Public Health, American Public Health Association, vol. 96(3), pages 538-546.
    6. Joanna Sale & Lynne Lohfeld & Kevin Brazil, 2002. "Revisiting the Quantitative-Qualitative Debate: Implications for Mixed-Methods Research," Quality & Quantity: International Journal of Methodology, Springer, vol. 36(1), pages 43-53, February.
    7. Trochim, William M. K., 1989. "An introduction to concept mapping for planning and evaluation," Evaluation and Program Planning, Elsevier, vol. 12(1), pages 1-16, January.
    8. Trochim, William M. K., 1989. "Concept mapping : Soft science or hard art?," Evaluation and Program Planning, Elsevier, vol. 12(1), pages 87-110, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kinga Varga & Ciaran MacDonncha & Laurence Blondel & Enrico Bozzano & Fabrice Burlot & Rute Costa & Nadine Debois & Dominique Delon & Antonio Figueiredo & Joerg Foerster & Masar Gjaka & Carlos Gonçalv, 2021. "Collective conceptualization of parental support of dual career athletes: The EMPATIA framework," PLOS ONE, Public Library of Science, vol. 16(9), pages 1-23, September.
    2. Goldman, Alyssa W. & Kane, Mary, 2014. "Concept mapping and network analysis: An analytic approach to measure ties among constructs," Evaluation and Program Planning, Elsevier, vol. 47(C), pages 9-17.
    3. Sofia Patsali, 2019. "Opening the black box of university-suppliers' co-invention: some field study evidence," Working Papers of BETA 2019-46, Bureau d'Economie Théorique et Appliquée, UDS, Strasbourg.
    4. Rosas, Scott R. & Ridings, John W., 2017. "The use of concept mapping in measurement development and evaluation: Application and future directions," Evaluation and Program Planning, Elsevier, vol. 60(C), pages 265-276.
    5. Péladeau, Normand & Dagenais, Christian & Ridde, Valéry, 2017. "Concept mapping internal validity: A case of misconceived mapping?," Evaluation and Program Planning, Elsevier, vol. 62(C), pages 56-63.
    6. Achim Goerres & Katrin Prinzen, 2012. "Using mixed methods for the analysis of individuals: a review of necessary and sufficient conditions and an application to welfare state attitudes," Quality & Quantity: International Journal of Methodology, Springer, vol. 46(2), pages 415-450, February.
    7. Johnsen, Jeffrey A. & Biegel, David E. & Shafran, Robert, 2000. "Concept mapping in mental health: uses and adaptations," Evaluation and Program Planning, Elsevier, vol. 23(1), pages 67-75, February.
    8. Orsi, Rebecca, 2017. "Use of multiple cluster analysis methods to explore the validity of a community outcomes concept map," Evaluation and Program Planning, Elsevier, vol. 60(C), pages 277-283.
    9. Brown, Jason D. & Sigvaldason, Nadine & Bednar, Lisa M., 2005. "Foster parent perceptions of placement needs for children with a fetal alcohol spectrum disorder," Children and Youth Services Review, Elsevier, vol. 27(3), pages 309-327, March.
    10. Maier, Konradin & Stix, Volker, 2013. "A semi-automated approach for structuring multi criteria decision problems," European Journal of Operational Research, Elsevier, vol. 225(3), pages 487-496.
    11. Mary Ganis & John Minnery & Iderlina Mateo-Babiano, 2016. "Planning people–places: A small world network paradigm for masterplanning with people in mind," Environment and Planning B, , vol. 43(6), pages 1075-1095, November.
    12. Alejandro Alvarado-Herrera & Enrique Bigne & Joaquín Aldas-Manzano & Rafael Curras-Perez, 2017. "A Scale for Measuring Consumer Perceptions of Corporate Social Responsibility Following the Sustainable Development Paradigm," Journal of Business Ethics, Springer, vol. 140(2), pages 243-262, January.
    13. Ryan, Scott & Wiles, Debra & Cash, Scottye & Siebert, Carl, 2005. "Risk assessments: empirically supported or values driven?," Children and Youth Services Review, Elsevier, vol. 27(2), pages 213-225, February.
    14. Michalski, Greg V. & Cousins, J. Bradley, 2000. "Differences in stakeholder perceptions about training evaluation: a concept mapping/pattern matching investigation," Evaluation and Program Planning, Elsevier, vol. 23(2), pages 211-230, May.
    15. Trochim, William M., 2017. "Hindsight is 20/20: Reflections on the evolution of concept mapping," Evaluation and Program Planning, Elsevier, vol. 60(C), pages 176-185.
    16. McLinden, Daniel, 2017. "And then the internet happened: Thoughts on the future of concept mapping," Evaluation and Program Planning, Elsevier, vol. 60(C), pages 293-300.
    17. Donnelly, James P., 2017. "A systematic review of concept mapping dissertations," Evaluation and Program Planning, Elsevier, vol. 60(C), pages 186-193.
    18. McLinden, Daniel, 2013. "Concept maps as network data: Analysis of a concept map using the methods of social network analysis," Evaluation and Program Planning, Elsevier, vol. 36(1), pages 40-48.
    19. Scott R. Rosas, 2017. "Multi-map comparison for group concept mapping: an approach for examining conceptual congruence through spatial correspondence," Quality & Quantity: International Journal of Methodology, Springer, vol. 51(6), pages 2421-2439, November.
    20. Brown, Jason D. & Rodger, Susan, 2009. "Children with disabilities: Problems faced by foster parents," Children and Youth Services Review, Elsevier, vol. 31(1), pages 40-46, January.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:epplan:v:35:y:2012:i:2:p:236-245. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/evalprogplan .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.