IDEAS home Printed from https://ideas.repec.org/a/sae/evarev/v14y1990i4p347-373.html
   My bibliography  Save this article

Assessing the Validity of Randomized Field Experiments

Author

Listed:
  • Michael L. Dennis

    (Center for Social Research and Policy Analysis Research Triangle Institute)

Abstract

Randomized field experiments are often logistical failures. They are particularly difficult to implement when the intervention involves a large number of components or service providers. Although surmountable, potential methodological problems with using randomized experiments to evaluate intervention programs under field (i.e., real world) conditions must be anticipated and resolved for the experiment to succeed. This article examines six potential major problems: treatment dilution, treatment contamination or confounding, inaccurate caseflow and power estimates, violations of the random assignment process, changes in the environmental context, and changes in the treatment regimens. It describes the general problem, several methodological developments related to them, and how the authors are attempting to deal with these problems in an experiment at the Research Triangle Institute. Developing designs and implementation strategies to deal with these six problems is crucial to extending the usefulness of randomized field experiments to evaluate social policies and experiments.

Suggested Citation

  • Michael L. Dennis, 1990. "Assessing the Validity of Randomized Field Experiments," Evaluation Review, , vol. 14(4), pages 347-373, August.
    • Handle: RePEc:sae:evarev:v:14:y:1990:i:4:p:347-373
    DOI: 10.1177/0193841X9001400402
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0193841X9001400402
    Download Restriction: no
    File URL: https://libkey.io/10.1177/0193841X9001400402?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Samuel Greenhouse & Seymour Geisser, 1959. "On methods in the analysis of profile data," Psychometrika, Springer;The Psychometric Society, vol. 24(2), pages 95-112, June.
    2. Scott, Anne G. & Sechrest, Lee, 1989. "Strength of theory and theory of strength," Evaluation and Program Planning, Elsevier, vol. 12(4), pages 329-336, January.
    3. Raymond Collier & Frank Baker & Garrett Mandeville & Thomas Hayes, 1967. "Estimates of test size for several test procedures based on conventional variance ratios in the repeated measures design," Psychometrika, Springer;The Psychometric Society, vol. 32(3), pages 339-353, September.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Paula Fernández García & Guillermo Vallejo & Pablo Livacic-Rojas & Javier Herrero & Marcelino Cuesta, 2010. "Compative robustness of six tests in repeated measures designs with specified departures from sphericity," Quality & Quantity: International Journal of Methodology, Springer, vol. 44(2), pages 289-301, February.
    2. Huynh Huynh, 1978. "Some approximate tests for repeated measurement designs," Psychometrika, Springer;The Psychometric Society, vol. 43(2), pages 161-175, June.
    3. Robert Boik, 1981. "A priori tests in repeated measures designs: Effects of nonsphericity," Psychometrika, Springer;The Psychometric Society, vol. 46(3), pages 241-255, September.
    4. Norma Naima Rüther & Marco Tettamanti & Stefano F Cappa & Christian Bellebaum, 2014. "Observed Manipulation Enhances Left Fronto-Parietal Activations in the Processing of Unfamiliar Tools," PLOS ONE, Public Library of Science, vol. 9(6), pages 1-9, June.
    5. Lifshitz, Chen Chana, 2017. "Fostering employability among youth at-risk in a multi-cultural context: Insights from a pilot intervention program," Children and Youth Services Review, Elsevier, vol. 76(C), pages 20-34.
    6. Lena Ulm & Dorota Wohlrapp & Marcus Meinzer & Robert Steinicke & Alexej Schatz & Petra Denzler & Juliane Klehmet & Christian Dohle & Michael Niedeggen & Andreas Meisel & York Winter, 2013. "A Circle-Monitor for Computerised Assessment of Visual Neglect in Peripersonal Space," PLOS ONE, Public Library of Science, vol. 8(12), pages 1-10, December.
    7. Chung-Wei Kuo, 2021. "Can We Return to Our Normal Life When the Pandemic Is under Control? A Preliminary Study on the Influence of COVID-19 on the Tourism Characteristics of Taiwan," Sustainability, MDPI, vol. 13(17), pages 1-17, August.
    8. Giorgia Cona & Giorgio Arcara & Vincenza Tarantino & Patrizia Silvia Bisiacchi, 2012. "Electrophysiological Correlates of Strategic Monitoring in Event-Based and Time-Based Prospective Memory," PLOS ONE, Public Library of Science, vol. 7(2), pages 1-9, February.
    9. Alessandro Grecucci & Simone Sulpizio & Elisa Tommasello & Francesco Vespignani & Remo Job, 2019. "Seeing emotions, reading emotions: Behavioral and ERPs evidence of the regulation of pictures and words," PLOS ONE, Public Library of Science, vol. 14(5), pages 1-21, May.
    10. Qian Shang & Guanxiong Pei & Jia Jin & Wuke Zhang & Yuran Wang & Xiaoyi Wang, 2018. "ERP evidence for consumer evaluation of copycat brands," PLOS ONE, Public Library of Science, vol. 13(2), pages 1-13, February.
    11. Sylvan Wallenstein & Joseph Fleiss, 1979. "Repeated measurements analysis of variance when the correlations have a certain pattern," Psychometrika, Springer;The Psychometric Society, vol. 44(2), pages 229-233, June.
    12. Kent, Douglas R. & Donaldson, Stewart I. & Wyrick, Phelan A. & Smith, Peggy J., 2000. "Evaluating criminal justice programs designed to reduce crime by targeting repeat gang offenders," Evaluation and Program Planning, Elsevier, vol. 23(1), pages 115-124, February.
    13. Friedrich, Sarah & Brunner, Edgar & Pauly, Markus, 2017. "Permuting longitudinal data in spite of the dependencies," Journal of Multivariate Analysis, Elsevier, vol. 153(C), pages 255-265.
    14. Samuel S. Komorita, 1978. "Evaluating Coalition Theories: Some Indices," Journal of Conflict Resolution, Peace Science Society (International), vol. 22(4), pages 691-706, December.
    15. M. Carme Ruiz De Villa & M. Salome & E. Cabral & Eduardo Escrich Escriche & Montse Solanas, 1999. "A non-parametric regression approach to repeated measures analysis in cancer experiments," Journal of Applied Statistics, Taylor & Francis Journals, vol. 26(5), pages 601-611.
    16. Martin Worm & Frank Kirschbaum & Gerhard von der Emde, 2017. "Social interactions between live and artificial weakly electric fish: Electrocommunication and locomotor behavior of Mormyrus rume proboscirostris towards a mobile dummy fish," PLOS ONE, Public Library of Science, vol. 12(9), pages 1-31, September.
    17. Cooksy, Leslie J. & Gill, Paige & Kelly, P. Adam, 2001. "The program logic model as an integrative framework for a multimethod evaluation," Evaluation and Program Planning, Elsevier, vol. 24(2), pages 119-128, May.
    18. Manuel Zaepffel & Romain Trachel & Bjørg Elisabeth Kilavik & Thomas Brochier, 2013. "Modulations of EEG Beta Power during Planning and Execution of Grasping Movements," PLOS ONE, Public Library of Science, vol. 8(3), pages 1-10, March.
    19. Li, Jing, 2023. "Longitudinal interplays between basic psychological need satisfaction and sleep among older adults in China," Social Science & Medicine, Elsevier, vol. 323(C).
    20. Livia Taverna & Marta Tremolada & Liliana Dozza & Renata Zanin Scaratti & Domahs Ulrike & Carlo Lallo & Barbara Tosetto, 2020. "Who Benefits from An Intervention Program on Foundational Skills for Handwriting Addressed to Kindergarten Children and First Graders?," IJERPH, MDPI, vol. 17(6), pages 1-24, March.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:evarev:v:14:y:1990:i:4:p:347-373. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.