IDEAS home Printed from https://ideas.repec.org/p/cwl/cwldpp/2080.html
   My bibliography  Save this paper

Research Design Meets Market Design: Using Centralized Assignment for Impact Evaluation

Author

Listed:

Abstract

A growing number of school districts use centralized assignment mechanisms to allocate school seats in a manner that reflects student preferences and school priorities. Many of these assignment schemes use lotteries to ration seats when schools are oversubscribed. The resulting random assignment opens the door to credible quasi-experimental research designs for the evaluation of school effectiveness. Yet the question of how best to separate the lottery-generated variation integral to such designs from non-random preferences and priorities remains open. This paper develops easily-implemented empirical strategies that fully exploit the random assignment embedded in a wide class of mechanisms, while also revealing why seats are randomized at one school but not another. We use these methods to evaluate charter schools in Denver, one of a growing number of districts that combine charter and traditional public schools in a unified assignment system. The resulting estimates show large achievement gains from charter school attendance. Our approach generates efficiency gains over ad hoc methods, such as those that focus on schools ranked first, while also identifying a more representative average causal effect. We also show how to use centralized assignment mechanisms to identify causal effects in models with multiple school sectors.

Suggested Citation

  • Atila Abdulkadiroglu & Joshua D. Angrist & Yusuke Narita & Parag A. Pathak, 2017. "Research Design Meets Market Design: Using Centralized Assignment for Impact Evaluation," Cowles Foundation Discussion Papers 2080, Cowles Foundation for Research in Economics, Yale University.
  • Handle: RePEc:cwl:cwldpp:2080
    as

    Download full text from publisher

    File URL: https://cowles.yale.edu/sites/default/files/files/pub/d20/d2080.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Luc Behaghel & Bruno Crépon & Marc Gurgand, 2013. "Robustness of the encouragement design in a two-treatment randomized control trial," Working Papers halshs-00834169, HAL.
    2. Philip M. Gleason & Christina Clark Tuttle & Brian Gill & Ira Nichols-Barrer & Bing-ru Teh, 2014. "Do KIPP Schools Boost Student Achievement?," Education Finance and Policy, MIT Press, vol. 9(1), pages 36-58, January.
    3. Joshua D. Angrist & Parag A. Pathak & Christopher R. Walters, 2013. "Explaining Charter School Effectiveness," American Economic Journal: Applied Economics, American Economic Association, vol. 5(4), pages 1-27, October.
    4. Yeon-Koo Che & Fuhito Kojima, 2010. "Asymptotic Equivalence of Probabilistic Serial and Random Priority Mechanisms," Econometrica, Econometric Society, vol. 78(5), pages 1625-1672, September.
    5. Adrienne M. Lucas & Isaac M. Mbiti, 2014. "Effects of School Quality on Student Achievement: Discontinuity Evidence from Kenya," American Economic Journal: Applied Economics, American Economic Association, vol. 6(3), pages 234-263, July.
    6. Atila Abdulkadiro?lu & Yeon-Koo Che & Yosuke Yasuda, 2015. "Expanding "Choice" in School Choice," American Economic Journal: Microeconomics, American Economic Association, vol. 7(1), pages 1-42, February.
    7. Simon Burgess & Ellen Greaves & Anna Vignoles & Deborah Wilson, 2015. "What Parents Want: School Preferences and School Choice," Economic Journal, Royal Economic Society, vol. 125(587), pages 1262-1289, September.
    8. Atila Abdulkadiroglu & Tayfun Sönmez, 2003. "School Choice: A Mechanism Design Approach," American Economic Review, American Economic Association, vol. 93(3), pages 729-747, June.
    9. Joshua D. Angrist & Peter D. Hull & Parag A. Pathak & Christopher R. Walters, 2017. "Leveraging Lotteries for School Value-Added: Testing and Estimation," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 132(2), pages 871-919.
    10. Atila Abdulkadiroglu & Tayfun Sonmez, 1998. "Random Serial Dictatorship and the Core from Random Endowments in House Allocation Problems," Econometrica, Econometric Society, vol. 66(3), pages 689-702, May.
    11. Atila Abdulkadiroğlu & Joshua D. Angrist & Peter D. Hull & Parag A. Pathak, 2016. "Charters without Lotteries: Testing Takeovers in New Orleans and Boston," American Economic Review, American Economic Association, vol. 106(7), pages 1878-1920, July.
    12. Caterina Calsamiglia & Chao Fu & Maia Güell, 2014. "Structural Estimation of a Model of School Choices: the Boston Mechanism vs. Its Alternatives," Working Papers 811, Barcelona School of Economics.
    13. repec:oup:restud:v:84:y::i:1:p:444-463. is not listed on IDEAS
    14. Ergin, Haluk & Sonmez, Tayfun, 2006. "Games of school choice under the Boston mechanism," Journal of Public Economics, Elsevier, vol. 90(1-2), pages 215-237, January.
    15. Alberto Abadie & Susan Athey & Guido W. Imbens & Jeffrey M. Wooldridge, 2014. "Finite Population Causal Standard Errors," NBER Working Papers 20325, National Bureau of Economic Research, Inc.
    16. Joshua Angrist & Jinyong Hahn, 2004. "When to Control for Covariates? Panel Asymptotics for Estimates of Treatment Effects," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 58-72, February.
    17. Atila Abdulkadiroğlu & Joshua D. Angrist & Yusuke Narita & Parag A. Pathak, 2017. "Research Design Meets Market Design: Using Centralized Assignment for Impact Evaluation," Econometrica, Econometric Society, vol. 85, pages 1373-1432, September.
    18. Atila Abdulkadroǧlu & Joshua D. Angrist & Yusuke Narita & Parag A. Pathak & Roman A. Zarate, 2017. "Regression Discontinuity in Serial Dictatorship: Achievement Effects at Chicago's Exam Schools," American Economic Review, American Economic Association, vol. 107(5), pages 240-245, May.
    19. Joshua D. Angrist & Sarah R. Cohodes & Susan M. Dynarski & Parag A. Pathak & Christopher R. Walters, 2016. "Stand and Deliver: Effects of Boston's Charter High Schools on College Preparation, Entry, and Choice," Journal of Labor Economics, University of Chicago Press, vol. 34(2), pages 275-318.
    20. Howard S. Bloom & Rebecca Unterman, 2014. "Can Small High Schools of Choice Improve Educational Prospects for Disadvantaged Students?," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 33(2), pages 290-319, March.
    21. Julie Berry Cullen & Brian A Jacob & Steven Levitt, 2006. "The Effect of School Choice on Participants: Evidence from Randomized Lotteries," Econometrica, Econometric Society, vol. 74(5), pages 1191-1230, September.
    22. , A. & ,, 2011. "Lotteries in student assignment: An equivalence result," Theoretical Economics, Econometric Society, vol. 6(1), January.
    23. Newey, Whitney K, 1990. "Efficient Instrumental Variables Estimation of Nonlinear Models," Econometrica, Econometric Society, vol. 58(4), pages 809-837, July.
    24. Lars-Gunnar Svensson, 1999. "Strategy-proof allocation of indivisible goods," Social Choice and Welfare, Springer;The Society for Social Choice and Welfare, vol. 16(4), pages 557-567.
    25. Keisuke Hirano & Guido W. Imbens & Geert Ridder, 2003. "Efficient Estimation of Average Treatment Effects Using the Estimated Propensity Score," Econometrica, Econometric Society, vol. 71(4), pages 1161-1189, July.
    26. Cristian Pop-Eleches & Miguel Urquiola, 2013. "Going to a Better School: Effects and Behavioral Responses," American Economic Review, American Economic Association, vol. 103(4), pages 1289-1324, June.
    27. Monique De Haan & Pieter A. Gautier & Hessel Oosterbeek & Bas van der Klaauw, 2023. "The Performance of School Assignment Mechanisms in Practice," Journal of Political Economy, University of Chicago Press, vol. 131(2), pages 388-455.
    28. Shu Yang & Guido W. Imbens & Zhanglin Cui & Douglas E. Faries & Zbigniew Kadziola, 2016. "Propensity score matching and subclassification in observational studies with multi‐level treatments," Biometrics, The International Biometric Society, vol. 72(4), pages 1055-1065, December.
    29. Atila Abdulkadiroğlu & Joshua D. Angrist & Susan M. Dynarski & Thomas J. Kane & Parag A. Pathak, 2011. "Accountability and Flexibility in Public Schools: Evidence from Boston's Charters And Pilots," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 126(2), pages 699-748.
    30. Frolich, Markus, 2007. "Nonparametric IV estimation of local average treatment effects with covariates," Journal of Econometrics, Elsevier, vol. 139(1), pages 35-75, July.
    31. Philip M. Gleason & Christina Clark Tuttle & Brian Gill & Ira Nichols-Barrer & Bing-ru Teh, 2014. "Do KIPP Schools Boost Student Achievement?," Mathematica Policy Research Reports 025c1b3af1c040ecbedfbce45, Mathematica Policy Research.
    32. Bing-ru Teh & Moria McCullough & Brian P. Gill, "undated". "Student Achievement in New York City Middle Schools Affiliated with Achievement First and Uncommon Schools," Mathematica Policy Research Reports 9b7646f5c8ae4067840b186c5, Mathematica Policy Research.
    33. Nikhil Agarwal & Paulo Somaini, 2018. "Demand Analysis Using Strategic Reports: An Application to a School Choice Mechanism," Econometrica, Econometric Society, vol. 86(2), pages 391-444, March.
    34. Atila Abdulkadiroğlu & Joshua Angrist & Parag Pathak, 2014. "The Elite Illusion: Achievement Effects at Boston and New York Exam Schools," Econometrica, Econometric Society, vol. 82(1), pages 137-196, January.
    35. Imbens, Guido W & Angrist, Joshua D, 1994. "Identification and Estimation of Local Average Treatment Effects," Econometrica, Econometric Society, vol. 62(2), pages 467-475, March.
    36. Umut M. Dur & Scott Duke Kominers & Parag A. Pathak & Tayfun Sönmez, 2013. "The Demise of Walk Zones in Boston: Priorities vs. Precedence in School Choice," NBER Working Papers 18981, National Bureau of Economic Research, Inc.
    37. Eduardo M. Azevedo & Jacob D. Leshno, 2016. "A Supply and Demand Framework for Two-Sided Matching Markets," Journal of Political Economy, University of Chicago Press, vol. 124(5), pages 1235-1268.
    38. Eric Budish, 2011. "The Combinatorial Assignment Problem: Approximate Competitive Equilibrium from Equal Incomes," Journal of Political Economy, University of Chicago Press, vol. 119(6), pages 1061-1103.
    39. Jinyong Hahn, 1998. "On the Role of the Propensity Score in Efficient Semiparametric Estimation of Average Treatment Effects," Econometrica, Econometric Society, vol. 66(2), pages 315-332, March.
    40. Abadie A., 2002. "Bootstrap Tests for Distributional Treatment Effects in Instrumental Variable Models," Journal of the American Statistical Association, American Statistical Association, vol. 97, pages 284-292, March.
    41. Umut Mert Dur & Parag A. Pathak & Tayfun Sönmez, 2016. "Explicit vs. Statistical Preferential Treatment in Affirmative Action: Theory and Evidence from Chicago’s Exam Schools," Boston College Working Papers in Economics 906, Boston College Department of Economics.
    42. Vaart,A. W. van der, 2000. "Asymptotic Statistics," Cambridge Books, Cambridge University Press, number 9780521784504, September.
    43. Imbens,Guido W. & Rubin,Donald B., 2015. "Causal Inference for Statistics, Social, and Biomedical Sciences," Cambridge Books, Cambridge University Press, number 9780521885881, September.
    44. C. Kirabo Jackson, 2010. "Do Students Benefit from Attending Better Schools? Evidence from Rule-based Student Assignments in Trinidad and Tobago," Economic Journal, Royal Economic Society, vol. 120(549), pages 1399-1429, December.
    45. Fuhito Kojima & Parag A. Pathak, 2009. "Incentives and Stability in Large Two-Sided Matching Markets," American Economic Review, American Economic Association, vol. 99(3), pages 608-627, June.
    46. Han Hong & Denis Nekipelov, 2010. "Semiparametric efficiency in nonlinear LATE models," Quantitative Economics, Econometric Society, vol. 1(2), pages 279-304, November.
    47. repec:mpr:mprres:6698 is not listed on IDEAS
    48. Will Dobbie & Roland G. Fryer, 2011. "Are High-Quality Schools Enough to Increase Achievement among the Poor? Evidence from the Harlem Children's Zone," American Economic Journal: Applied Economics, American Economic Association, vol. 3(3), pages 158-187, July.
    49. repec:mpr:mprres:8002 is not listed on IDEAS
    50. Atila Abdulkadiroğlu & Weiwei Hu & Parag A. Pathak, 2013. "Small High Schools and Student Achievement: Lottery-Based Evidence from New York City," NBER Working Papers 19576, National Bureau of Economic Research, Inc.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yusuke Narita, 2020. "A Theory of Quasi-Experimental Evaluation of School Quality," Working Papers 2020-085, Human Capital and Economic Opportunity Working Group.
    2. Yusuki Narita, 2016. "(Non)Randomization: A Theory of Quasi-Experimental Evaluation of School Quality," Cowles Foundation Discussion Papers 2056R, Cowles Foundation for Research in Economics, Yale University, revised Aug 2017.
    3. Atı̇la Abdulkadı̇roğlu & Joshua D. Angrist & Yusuke Narita & Parag Pathak, 2022. "Breaking Ties: Regression Discontinuity Design Meets Market Design," Econometrica, Econometric Society, vol. 90(1), pages 117-151, January.
    4. Gabrielle Fack & Julien Grenet & Yinghua He, 2019. "Beyond Truth-Telling: Preference Estimation with Centralized School Choice and College Admissions," American Economic Review, American Economic Association, vol. 109(4), pages 1486-1529, April.
    5. Angrist, Joshua D. & Pathak, Parag A. & Zarate, Roman A., 2023. "Choice and consequence: Assessing mismatch at Chicago exam schools," Journal of Public Economics, Elsevier, vol. 223(C).
    6. Yusuke Narita, 2021. "A Theory of Quasi-Experimental Evaluation of School Quality," Management Science, INFORMS, vol. 67(8), pages 4982-5010, August.
    7. Figlio, D. & Karbownik, K. & Salvanes, K.G., 2016. "Education Research and Administrative Data," Handbook of the Economics of Education,, Elsevier.
    8. Clément de Chaisemartin & Luc Behaghel, 2020. "Estimating the Effect of Treatments Allocated by Randomized Waiting Lists," Econometrica, Econometric Society, vol. 88(4), pages 1453-1477, July.
    9. Marco Ovidi, 2021. "Parents know better: primary school choice and student achievement in London," Working Papers 919, Queen Mary University of London, School of Economics and Finance.
    10. Atila Abdulkadiroğlu & Nikhil Agarwal & Parag A. Pathak, 2015. "The Welfare Effects of Coordinated Assignment: Evidence from the NYC HS Match," NBER Working Papers 21046, National Bureau of Economic Research, Inc.
    11. Oosterbeek, Hessel & Ruijs, Nienke & de Wolf, Inge, 2023. "Heterogeneous effects of comprehensive vs. single-track academic schools: Evidence from admission lotteries," Economics of Education Review, Elsevier, vol. 93(C).
    12. Diether W Beuermann & C Kirabo Jackson & Laia Navarro-Sola & Francisco Pardo, 2023. "What is a Good School, and Can Parents Tell? Evidence on the Multidimensionality of School Output," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 90(1), pages 65-101.
    13. Lars Kirkebøen & Edwin Leuven & Magne Mogstad, 2014. "Field of Study, Earnings, and Self-Selection," NBER Working Papers 20816, National Bureau of Economic Research, Inc.
    14. Ha, Wei & Kang, Le & Song, Yang, 2020. "College matching mechanisms and matching stability: Evidence from a natural experiment in China," Journal of Economic Behavior & Organization, Elsevier, vol. 175(C), pages 206-226.
    15. Thierry Magnac, 2018. "Quels étudiants pour quelles universités ? Analyses empiriques de mécanismes d’allocation centralisée," Revue économique, Presses de Sciences-Po, vol. 69(5), pages 683-708.
    16. Atila Abdulkadiroğlu & Weiwei Hu & Parag A. Pathak, 2013. "Small High Schools and Student Achievement: Lottery-Based Evidence from New York City," NBER Working Papers 19576, National Bureau of Economic Research, Inc.
    17. Joshua D. Angrist, 2022. "Empirical Strategies in Economics: Illuminating the Path From Cause to Effect," Econometrica, Econometric Society, vol. 90(6), pages 2509-2539, November.
    18. Ashlagi, Itai & Nikzad, Afshin & Romm, Assaf, 2019. "Assigning more students to their top choices: A comparison of tie-breaking rules," Games and Economic Behavior, Elsevier, vol. 115(C), pages 167-187.
    19. Akbarpour, Mohammad & Kapor, Adam & Neilson, Christopher & van Dijk, Winnie & Zimmerman, Seth, 2022. "Centralized School choice with unequal outside options," Journal of Public Economics, Elsevier, vol. 210(C).
    20. Ketel, Nadine & Oosterbeek, Hessel & Sovago, Sandor & van der Klaauw, Bas, 2023. "The (Un)Importance of School Assignment," IZA Discussion Papers 16591, Institute of Labor Economics (IZA).

    More about this item

    Keywords

    Matching Market Design; Natural Experiment; Program Evaluation; Random Assignment; Quasi-Experimental Research Design; School Effectiveness; Charter Schools;
    All these keywords.

    JEL classification:

    • C26 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Instrumental Variables (IV) Estimation
    • D47 - Microeconomics - - Market Structure, Pricing, and Design - - - Market Design
    • I20 - Health, Education, and Welfare - - Education - - - General

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:cwl:cwldpp:2080. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Brittany Ladd (email available below). General contact details of provider: https://edirc.repec.org/data/cowleus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.