IDEAS home Printed from https://ideas.repec.org/a/eee/epplan/v91y2022ics0149718922000106.html
   My bibliography  Save this article

What does it mean for an evaluation to be ‘valid’? A critical synthesis of evaluation literature

Author

Listed:
  • Downes, Jenni
  • Gullickson, Amy M.

Abstract

Validity is both a critical foundation for, and widely contested concept in the field of program evaluation. Given the wide and divergent treatment of the term, this article sought to enhance the field’s understanding and application of validity by systematically identifying, collating and integrating existing conceptions of validity. A critical interpretive synthesis approach was taken to combine the best elements of systematic literature searching with integrative review. We found that conceptions of validity could be roughly collated into three categories pertaining to the evaluation research, evaluative judgement and reporting, and beyond the evaluation proper. The result was a wide-ranging map of validity concepts against the various stages of the evaluation process. This made clear several gaps in the validity thinking, resulting in the development of comprehensive but not yet complete framework that weaves together related aspects identified into a cohesive whole. It is our hope that this initial iteration of a cohesive validity framework will be the starting point for a cooperative effort to improve professional evaluation practice.

Suggested Citation

  • Downes, Jenni & Gullickson, Amy M., 2022. "What does it mean for an evaluation to be ‘valid’? A critical synthesis of evaluation literature," Evaluation and Program Planning, Elsevier, vol. 91(C).
    • Handle: RePEc:eee:epplan:v:91:y:2022:i:c:s0149718922000106
    DOI: 10.1016/j.evalprogplan.2022.102056
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0149718922000106
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.evalprogplan.2022.102056?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Arbour, Ghislain, 2020. "Teaching programme evaluation: A problem of knowledge," Evaluation and Program Planning, Elsevier, vol. 83(C).
    2. Benjamin Saunders & Julius Sim & Tom Kingstone & Shula Baker & Jackie Waterfield & Bernadette Bartlam & Heather Burroughs & Clare Jinks, 2018. "Saturation in qualitative research: exploring its conceptualization and operationalization," Quality & Quantity: International Journal of Methodology, Springer, vol. 52(4), pages 1893-1907, July.
    3. Patton, Michael Quinn, 1989. "A context and boundaries for a theory-driven approach to validity," Evaluation and Program Planning, Elsevier, vol. 12(4), pages 375-377, January.
    4. Palumbo, Dennis J. & Oliverio, Annamarie, 1989. "Implementation theory and the theory-driven approach to validity," Evaluation and Program Planning, Elsevier, vol. 12(4), pages 337-344, January.
    5. Chen, Huey-Tsyh & Rossi, Peter H., 1987. "The theory-driven approach to validity," Evaluation and Program Planning, Elsevier, vol. 10(1), pages 95-103, January.
    6. Chen, Huey T., 2010. "The bottom-up approach to integrative validity: A new perspective for program evaluation," Evaluation and Program Planning, Elsevier, vol. 33(3), pages 205-214, August.
    7. Garaway, G. B., 1997. "Evaluation, validity, and values," Evaluation and Program Planning, Elsevier, vol. 20(1), pages 1-5, February.
    8. Gullickson, Amy M. & King, Jean A. & LaVelle, John M. & Clinton, Janet M., 2019. "The current state of evaluator education: A situation analysis and call to action," Evaluation and Program Planning, Elsevier, vol. 75(C), pages 20-30.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jennifer Tartaglia & Michelle McIntosh & Jonine Jancey & Jane Scott & Andrea Begley, 2021. "Exploring Feeding Practices and Food Literacy in Parents with Young Children from Disadvantaged Areas," IJERPH, MDPI, vol. 18(4), pages 1-18, February.
    2. Najwa Taghy & Linda Cambon & Caroline Boulliat & Olivier Aromatario & Claude Dussart, 2021. "Exploring the Determinants of Polypharmacy Prescribing and Dispensing Behaviours in Primary Care for the Elderly—Protocol for a Qualitative Study," IJERPH, MDPI, vol. 18(14), pages 1-10, July.
    3. Van Droogenbroeck, Ellen & Van Hove, Leo, 2020. "Intra-household task allocation in online grocery shopping: Together alone," Journal of Retailing and Consumer Services, Elsevier, vol. 56(C).
    4. Thuy Séran & Anne-Sophie Fernandez & Hervé Chappert, 2024. "Managing coopetition in multi-unit organizations: a management-control perspective," Review of Managerial Science, Springer, vol. 18(10), pages 2889-2924, October.
    5. Şahika Simsek-Cetinkaya & Simge Evrenol Ocal, 2023. "“Psychological Injuries Are Not Visible†: Experiences and Perceptions of Midwives and Nurses about Domestic Violence during Pregnancy," Clinical Nursing Research, , vol. 32(8), pages 1115-1123, November.
    6. Najwa Taghy & Viviane Ramel & Ana Rivadeneyra & Florence Carrouel & Linda Cambon & Claude Dussart, 2023. "Exploring the Determinants of Polypharmacy Prescribing and Dispensing Behaviors in Primary Care for the Elderly—Qualitative Study," IJERPH, MDPI, vol. 20(2), pages 1-17, January.
    7. Xudan Lin & Hong Zhu & Duo Yin, 2022. "Enhancing Rural Resilience in a Tea Town of China: Exploring Tea Farmers’ Knowledge Production for Tea Planting, Tea Processing and Tea Tasting," Land, MDPI, vol. 11(4), pages 1-18, April.
    8. Archibald, Thomas, 2015. "“They Just Know”: The epistemological politics of “evidence-based” non-formal education," Evaluation and Program Planning, Elsevier, vol. 48(C), pages 137-148.
    9. Silvia Scaramuzzi & Sara Gabellini & Giovanni Belletti & Andrea Marescotti, 2021. "Agrobiodiversity-Oriented Food Systems between Public Policies and Private Action: A Socio-Ecological Model for Sustainable Territorial Development," Sustainability, MDPI, vol. 13(21), pages 1-32, November.
    10. Urban, Jennifer Brown & Hargraves, Monica & Trochim, William M., 2014. "Evolutionary Evaluation: Implications for evaluators, researchers, practitioners, funders and the evidence-based program mandate," Evaluation and Program Planning, Elsevier, vol. 45(C), pages 127-139.
    11. Asiimire Donath & Medard Twinamatsiko & Johnson Atwiine & Dr. Nuwatuhaire Benard, 2024. "Women Productive Resource Ownership and their Contribution to the Changing Family Patterns in Ankole Sub-Region, Uganda," International Journal of Research and Scientific Innovation, International Journal of Research and Scientific Innovation (IJRSI), vol. 11(5), pages 844-855, May.
    12. Tongyu Meng & Jamie Newth & Christine Woods, 2022. "Ethical Sensemaking in Impact Investing: Reasons and Motives in the Chinese Renewable Energy Sector," Journal of Business Ethics, Springer, vol. 179(4), pages 1091-1117, September.
    13. Torvatn, Hans, 1999. "Using program theory models in evaluation of industrial modernization programs: three case studies," Evaluation and Program Planning, Elsevier, vol. 22(1), pages 73-82.
    14. Annette Peart & Virginia Lewis & Chris Barton & Grant Russell, 2020. "Healthcare professionals providing care coordination to people living with multimorbidity: An interpretative phenomenological analysis," Journal of Clinical Nursing, John Wiley & Sons, vol. 29(13-14), pages 2317-2328, July.
    15. Soo-Yong Shin & Eun-Ju Lim, 2021. "Clinical Work and Life of Mid-Career Male Nurses: A Qualitative Study," IJERPH, MDPI, vol. 18(12), pages 1-10, June.
    16. Fatoumata Fofana & Pat Bazeley & Antoine Regnault, 2020. "Applying a mixed methods design to test saturation for qualitative data in health outcomes research," PLOS ONE, Public Library of Science, vol. 15(6), pages 1-12, June.
    17. LaVelle, John M. & Davies, Randall, 2021. "Seeking consensus: Defining foundational concepts for a graduate level introductory program evaluation course," Evaluation and Program Planning, Elsevier, vol. 88(C).
    18. Alasdair Jones & Susan Parham, 2023. "Living in an Age-Friendly Community: Evidence from a Masterplanned Development in Southwest Sydney," IJERPH, MDPI, vol. 20(2), pages 1-21, January.
    19. Diego M. Coraiola & Robbin Derry, 2020. "Remembering to Forget: The Historic Irresponsibility of U.S. Big Tobacco," Journal of Business Ethics, Springer, vol. 166(2), pages 233-252, October.
    20. Shern, David L. & Trochim, William M. K. & LaComb, Christina A., 1995. "The use of concept mapping for assessing fidelity of model transfer: An example from psychiatric rehabilitation," Evaluation and Program Planning, Elsevier, vol. 18(2), pages 143-153.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:epplan:v:91:y:2022:i:c:s0149718922000106. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/evalprogplan .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.