IDEAS home Printed from https://ideas.repec.org/a/sae/evarev/v35y2011i5p490-522.html
   My bibliography  Save this article

Using Interviews to Understand the Assignment Mechanism in a Nonexperimental Study

Author

Listed:
  • Jordan H. Rickles

Abstract

Many inquiries regarding the causal effects of policies or programs are based on research designs where the treatment assignment process is unknown, and thus valid inferences depend on tenuous assumptions about the assignment mechanism. This article draws attention to the importance of understanding the assignment mechanism in policy and program evaluation studies, and illustrates how information collected through interviews can develop a richer understanding of the assignment mechanism. Focusing on the issue of student assignment to algebra in 8th grade, I show how a preliminary data collection effort aimed at understanding the assignment mechanism is particularly beneficial in multisite observational studies in education. The findings, based on ten interviews and administrative data from a large school district, draw attention to the often ignored heterogeneity in the assignment mechanism across schools. These findings likely extend beyond the current research project in question to related educational policy issues such as ability grouping, tracking, differential course taking, and curricular intensity, as well as other social programs in which the assignment mechanism can differ across sites.

Suggested Citation

  • Jordan H. Rickles, 2011. "Using Interviews to Understand the Assignment Mechanism in a Nonexperimental Study," Evaluation Review, , vol. 35(5), pages 490-522, October.
    • Handle: RePEc:sae:evarev:v:35:y:2011:i:5:p:490-522
    DOI: 10.1177/0193841X11428644
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0193841X11428644
    Download Restriction: no
    File URL: https://libkey.io/10.1177/0193841X11428644?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Shadish, William R. & Clark, M. H. & Steiner, Peter M., 2008. "Can Nonrandomized Experiments Yield Accurate Answers? A Randomized Experiment Comparing Random and Nonrandom Assignments," Journal of the American Statistical Association, American Statistical Association, vol. 103(484), pages 1334-1344.
    2. Donald B. Rubin, 2005. "Causal Inference Using Potential Outcomes: Design, Modeling, Decisions," Journal of the American Statistical Association, American Statistical Association, vol. 100, pages 322-331, March.
    3. Heckman, J.J. & Hotz, V.J., 1988. "Choosing Among Alternative Nonexperimental Methods For Estimating The Impact Of Social Programs: The Case Of Manpower Training," University of Chicago - Economics Research Center 88-12, Chicago - Economics Research Center.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Vivian C. Wong & Peter M. Steiner & Kylie L. Anglin, 2018. "What Can Be Learned From Empirical Evaluations of Nonexperimental Methods?," Evaluation Review, , vol. 42(2), pages 147-175, April.
    2. Lechner, Michael & Wunsch, Conny, 2013. "Sensitivity of matching-based program evaluations to the availability of control variables," Labour Economics, Elsevier, vol. 21(C), pages 111-121.
    3. Vivian C. Wong & Peter M. Steiner, 2018. "Designs of Empirical Evaluations of Nonexperimental Methods in Field Settings," Evaluation Review, , vol. 42(2), pages 176-213, April.
    4. Jordan H. Rickles & Michael Seltzer, 2014. "A Two-Stage Propensity Score Matching Strategy for Treatment Effect Estimation in a Multisite Observational Study," Journal of Educational and Behavioral Statistics, , vol. 39(6), pages 612-636, December.
    5. Goldberg, Matthew H., 2019. "How often does random assignment fail? Estimates and recommendations," OSF Preprints s2j4r, Center for Open Science.
    6. Cousineau, Martin & Verter, Vedat & Murphy, Susan A. & Pineau, Joelle, 2023. "Estimating causal effects with optimization-based methods: A review and empirical comparison," European Journal of Operational Research, Elsevier, vol. 304(2), pages 367-380.
    7. Robbiano, Simone, 2021. "The innovative impact of public research institutes: evidence from Italy," MPRA Paper 106386, University Library of Munich, Germany.
    8. Martin Cousineau & Vedat Verter & Susan A. Murphy & Joelle Pineau, 2022. "Estimating causal effects with optimization-based methods: A review and empirical comparison," Papers 2203.00097, arXiv.org.
    9. Ana Kolar & Peter M. Steiner, 2021. "The Role of Sample Size to Attain Statistically Comparable Groups – A Required Data Preprocessing Step to Estimate Causal Effects With Observational Data," Evaluation Review, , vol. 45(5), pages 195-227, October.
    10. Akanksha Negi, 2020. "Doubly weighted M-estimation for nonrandom assignment and missing outcomes," Papers 2011.11485, arXiv.org.
    11. James J. Heckman, 1991. "Randomization and Social Policy Evaluation Revisited," NBER Technical Working Papers 0107, National Bureau of Economic Research, Inc.
    12. Maximiliano Marzetti & Rok Spruk, 2023. "Long-Term Economic Effects of Populist Legal Reforms: Evidence from Argentina," Comparative Economic Studies, Palgrave Macmillan;Association for Comparative Economic Studies, vol. 65(1), pages 60-95, March.
    13. J. David Brown & John S. Earle, 2012. "Do SBA Loans Create Jobs? Estimates from Universal Panel Data and Longitudinal Matching Methods," Working Papers 12-27, Center for Economic Studies, U.S. Census Bureau.
    14. Salvatore Bimonte & Antonella D’Agostino, 2021. "Tourism development and residents’ well-being: Comparing two seaside destinations in Italy," Tourism Economics, , vol. 27(7), pages 1508-1525, November.
    15. Tymon Słoczyński, 2015. "The Oaxaca–Blinder Unexplained Component as a Treatment Effects Estimator," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 77(4), pages 588-604, August.
    16. Renata Baborska & Emilio Hernandez & Emiliano Magrini & Cristian Morales-Opazo, 2020. "The impact of financial inclusion on rural food security experience: A perspective from low-and middle-income countries," Review of Development Finance Journal, Chartered Institute of Development Finance, vol. 10(2), pages 1-18.
    17. Hämäläinen, Kari & Ollikainen, Virve, 2004. "Differential Effects of Active Labour Market Programmes in the Early Stages of Young People's Unemployment," Research Reports 115, VATT Institute for Economic Research.
    18. Lechner, Michael & Vazquez-Alvarez, Rosalia, 2003. "The Effect of Disability on Labour Market Outcomes in Germany: Evidence from Matching," IZA Discussion Papers 967, Institute of Labor Economics (IZA).
    19. Jayanta Bhattacharya & Frank Neuhauser & Robert T. Reville & Seth A. Seabury, 2010. "Evaluating Permanent Disability Ratings Using Empirical Data on Earnings Losses," Journal of Risk & Insurance, The American Risk and Insurance Association, vol. 77(1), pages 231-260, March.
    20. Krebs, Christopher P. & Lattimore, Pamela K. & Cowell, Alexander J. & Graham, Phillip, 2010. "Evaluating the Juvenile Breaking the Cycle Program's impact on recidivism," Journal of Criminal Justice, Elsevier, vol. 38(2), pages 109-117, March.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:evarev:v:35:y:2011:i:5:p:490-522. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.