IDEAS home Printed from https://ideas.repec.org/a/eee/epplan/v83y2020ics0149718920301622.html
   My bibliography  Save this article

Striking a Balance between Program-Specific and Portfolio-Level Evaluation: Lessons Learned from a Multi-Site Evaluation on the Texas-Mexico Border

Author

Listed:
  • Wolff, Lisa S.
  • Errichetti, Karen Sautter
  • Tapia Walker, Stephanie
  • Davis, Mary V.
  • Brodesky, Michelle Kuhns

Abstract

Funders are increasingly making strategic investments across multiple grantees, aiming for their portfolio to improve targeted outcomes in a specific issue area. To this end, funders might use multi-site evaluation (MSE) approaches to examine the impact of their collective investments. However, it is important to recognize that each program—and its own program evaluation—must be tailored to its setting, population, and local context to best meet the needs of its target population. Therefore, multi-site evaluations need to account for this complexity. This paper describes the Sí Texas project, a large initiative of eight grantees implementing different integrated behavioral health models to improve physical and mental health outcomes along the Texas-Mexico border. With over 4,200 MSE study participants, the evaluation for Sí Texas used a partnership-centered approach to both enhance the evidence base and build local organizational capacity. This paper describes this approach, the process of tailoring evaluation practices to the grantees’ context, and the challenge of balancing consistency at the grantee-level for the portfolio multi-site evaluation. Successes, challenges, and lessons learned related to study design, data collection, grantee partnership, and capacity building are discussed.

Suggested Citation

  • Wolff, Lisa S. & Errichetti, Karen Sautter & Tapia Walker, Stephanie & Davis, Mary V. & Brodesky, Michelle Kuhns, 2020. "Striking a Balance between Program-Specific and Portfolio-Level Evaluation: Lessons Learned from a Multi-Site Evaluation on the Texas-Mexico Border," Evaluation and Program Planning, Elsevier, vol. 83(C).
    • Handle: RePEc:eee:epplan:v:83:y:2020:i:c:s0149718920301622
    DOI: 10.1016/j.evalprogplan.2020.101858
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0149718920301622
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.evalprogplan.2020.101858?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Jordana Barton & Raquel Marquez & Emily Ryder Perlmeter & Elizabeth Sobel-Blum, 2015. "Las colonias in the 21st century: progress along the Texas-Mexico border," Monograph, Federal Reserve Bank of Dallas, number 00002.
    2. Jordana Barton & Emily Ryder Perlmeter & Elizabeth Sobel Blum & Raquel R. Márquez, 2015. "Las Colonias along the Texas-Mexico Border," Palgrave Macmillan Books, in: Pia M. Orrenius & Jesús Cañas & Michael Weiss (ed.), Ten-Gallon Economy, chapter 14, pages 213-229, Palgrave Macmillan.
    3. Fetterman, David M., 1994. "Steps of empowerment evaluation: From California to Cape Town," Evaluation and Program Planning, Elsevier, vol. 17(3), pages 305-313.
    4. Jordana Barton & Raquel Marquez & Emily Ryder Perlmeter & Elizabeth Sobel-Blum, 2015. "Las colonias in the 21st century: progress along the Texas-Mexico border," Perspectives 13, Federal Reserve Bank of Dallas.
    5. Stainbrook, Kristin & Penney, Darby & Elwyn, Laura, 2015. "The opportunities and challenges of multi-site evaluations: Lessons from the jail diversion and trauma recovery national cross-site evaluation," Evaluation and Program Planning, Elsevier, vol. 50(C), pages 26-35.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Green, Rex S. & Ellis, Peter T., 2008. "Foster youth evaluate the performance of group home services in California," Evaluation and Program Planning, Elsevier, vol. 31(2), pages 174-180, May.
    2. Lisa M. Vaughn & MaryAnn Lohmueller, 2014. "Calling All Stakeholders," Evaluation Review, , vol. 38(4), pages 336-355, August.
    3. Weber, Olaf & Ahmad, Adnan, 2014. "Empowerment Through Microfinance: The Relation Between Loan Cycle and Level of Empowerment," World Development, Elsevier, vol. 62(C), pages 75-87.
    4. Bob Townley & Stuart Wilks-Heeg, 1999. "Democratic evaluation: Putting principles into practice," Local Economy, London South Bank University, vol. 14(1), pages 27-42, May.
    5. Beardmore, Amy & Jones, Matthew & Seal, Joanne, 2023. "Outcome harvesting as a methodology for the retrospective evaluation of small-scale community development interventions," Evaluation and Program Planning, Elsevier, vol. 97(C).
    6. Dean Kyne, 2023. "A Bird’s-Eye View of Colonias Hosting Forgotten Americans and Their Community Resilience in the Rio Grande Valley," Geographies, MDPI, vol. 3(3), pages 1-18, July.
    7. José M. Díaz-Puente & José L. Yagüe & Ana Afonso, 2008. "Building Evaluation Capacity in Spain," Evaluation Review, , vol. 32(5), pages 478-506, October.
    8. Magro, Edurne & Wilson, James R., 2019. "Policy-mix evaluation: Governance challenges from new place-based innovation policies," Research Policy, Elsevier, vol. 48(10).
    9. Schalock, Robert L. & Bonham, Gordon S. & Marchand, Cristine B., 2000. "Consumer based quality of life assessment: a path model of perceived satisfaction," Evaluation and Program Planning, Elsevier, vol. 23(1), pages 77-87, February.
    10. Luis Arturo Pinzon‐Salcedo & María Alejandra Torres‐Cuello, 2022. "Systems thinking concepts within a collaborative programme evaluation methodology: The Hermes Programme evaluation," Systems Research and Behavioral Science, Wiley Blackwell, vol. 39(4), pages 708-722, July.
    11. Nanny Bressers & Lasse Gerrits, 2015. "A Complexity-Informed Approach to Evaluating National Knowledge and Innovation Programmes," Systems Research and Behavioral Science, Wiley Blackwell, vol. 32(1), pages 50-63, January.
    12. O’Leary, Susan, 2017. "Grassroots accountability promises in rights-based approaches to development: The role of transformative monitoring and evaluation in NGOs," Accounting, Organizations and Society, Elsevier, vol. 63(C), pages 21-41.
    13. Ryan, Katherine E. & Geissler, Barbara & Knell, Suzanne, 1996. "Progress and accountability in family literacy: Lessons from a collaborative approach," Evaluation and Program Planning, Elsevier, vol. 19(3), pages 263-272, August.
    14. Sudirman Zaid & La Hatani & Hayat Yusuf, 2020. "Building empowerment model of cocoa oil small-sized industries to increase value added," International Journal of Research in Business and Social Science (2147-4478), Center for the Strategic Studies in Business and Finance, vol. 9(4), pages 169-174, July.
    15. Anzoise, Valentina & Sardo, Stefania, 2016. "Dynamic systems and the role of evaluation: The case of the Green Communities project," Evaluation and Program Planning, Elsevier, vol. 54(C), pages 162-172.
    16. van den Eynde, Julie & Veno, Arthur & Hart, Alison, 2003. "They look good but don't work: a case study of global performance indicators in crime prevention," Evaluation and Program Planning, Elsevier, vol. 26(3), pages 237-248, August.
    17. Graeme Auld & Stefan Renckens & Benjamin Cashore, 2015. "Transnational private governance between the logics of empowerment and control," Regulation & Governance, John Wiley & Sons, vol. 9(2), pages 108-124, June.
    18. Ann Carpenter & Abram Lueders & Chris Thayer, 2017. "Informal Homeownership Issues: Tracking Contract for Deed Sales in the Southeast," FRB Atlanta Community and Economic Development Discussion Paper 2017-2, Federal Reserve Bank of Atlanta.
    19. Ramanath, Ramya, 2014. "Ethical implications of resource-limited evaluations: Lessons from an INGO in the eastern Democratic Republic of Congo," Evaluation and Program Planning, Elsevier, vol. 46(C), pages 25-37.
    20. McDonald, Diane, 1999. "Developing guidelines to enhance the evaluation of overseas development projects," Evaluation and Program Planning, Elsevier, vol. 22(2), pages 163-174, May.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:epplan:v:83:y:2020:i:c:s0149718920301622. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/evalprogplan .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.