IDEAS home Printed from https://ideas.repec.org/a/sae/evarev/v49y2025i2p237-269.html
   My bibliography  Save this article

Validity Evidence for an Observational Fidelity Measure to Inform Scale-Up of Evidence-Based Interventions

Author

Listed:
  • Pamela R. Buckley
  • Katie Massey Combs
  • Karen M. Drewelow
  • Brittany L. Hubler
  • Marion Amanda Lain

Abstract

As evidence-based interventions are scaled, fidelity of implementation, and thus effectiveness, often wanes. Validated fidelity measures can improve researchers’ ability to attribute outcomes to the intervention and help practitioners feel more confident in implementing the intervention as intended. We aim to provide a model for the validation of fidelity observation protocols to guide future research studying evidence-based interventions scaled-up under real-world conditions. We describe a process to build evidence of validity for items within the Session Review Form, an observational tool measuring fidelity to interactive drug prevention programs such as the Botvin LifeSkills Training program. Following Kane’s (2006) assumptions framework requiring that validity evidence be built across four areas (scoring, generalizability, extrapolation, and decision), confirmatory factor analysis supported the hypothesized two-factor structure measuring quality of delivery (seven items assessing how well the material is implemented) and participant responsiveness (three items evaluating how well the intervention is received), and measurement invariance tests suggested the structure held across grade level and schools serving different student populations. These findings provide some evidence supporting the extrapolation assumption, though additional research is warranted since a more complete overall depiction of the validity argument is needed to evaluate fidelity measures.

Suggested Citation

  • Pamela R. Buckley & Katie Massey Combs & Karen M. Drewelow & Brittany L. Hubler & Marion Amanda Lain, 2025. "Validity Evidence for an Observational Fidelity Measure to Inform Scale-Up of Evidence-Based Interventions," Evaluation Review, , vol. 49(2), pages 237-269, April.
  • Handle: RePEc:sae:evarev:v:49:y:2025:i:2:p:237-269
    DOI: 10.1177/0193841X241248864
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0193841X241248864
    Download Restriction: no

    File URL: https://libkey.io/10.1177/0193841X241248864?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:evarev:v:49:y:2025:i:2:p:237-269. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.