IDEAS home Printed from https://ideas.repec.org/a/wly/amposc/v61y2017i2p473-489.html
   My bibliography  Save this article

The Balance‐Sample Size Frontier in Matching Methods for Causal Inference

Author

Listed:
  • Gary King
  • Christopher Lucas
  • Richard A. Nielsen

Abstract

We propose a simplified approach to matching for causal inference that simultaneously optimizes balance (similarity between the treated and control groups) and matched sample size. Existing approaches either fix the matched sample size and maximize balance or fix balance and maximize sample size, leaving analysts to settle for suboptimal solutions or attempt manual optimization by iteratively tweaking their matching method and rechecking balance. To jointly maximize balance and sample size, we introduce the matching frontier, the set of matching solutions with maximum possible balance for each sample size. Rather than iterating, researchers can choose matching solutions from the frontier for analysis in one step. We derive fast algorithms that calculate the matching frontier for several commonly used balance metrics. We demonstrate this approach with analyses of the effect of sex on judging and job training programs that show how the methods we introduce can extract new knowledge from existing data sets.

Suggested Citation

  • Gary King & Christopher Lucas & Richard A. Nielsen, 2017. "The Balance‐Sample Size Frontier in Matching Methods for Causal Inference," American Journal of Political Science, John Wiley & Sons, vol. 61(2), pages 473-489, April.
  • Handle: RePEc:wly:amposc:v:61:y:2017:i:2:p:473-489
    DOI: 10.1111/ajps.12272
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/ajps.12272
    Download Restriction: no

    File URL: https://libkey.io/10.1111/ajps.12272?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Richard K. Crump & V. Joseph Hotz & Guido W. Imbens & Oscar A. Mitnik, 2009. "Dealing with limited overlap in estimation of average treatment effects," Biometrika, Biometrika Trust, vol. 96(1), pages 187-199.
    2. LaLonde, Robert J, 1986. "Evaluating the Econometric Evaluations of Training Programs with Experimental Data," American Economic Review, American Economic Association, vol. 76(4), pages 604-620, September.
    3. James J. Heckman & Hidehiko Ichimura & Petra E. Todd, 1997. "Matching As An Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 64(4), pages 605-654.
    4. Marco Caliendo & Sabine Kopeinig, 2008. "Some Practical Guidance For The Implementation Of Propensity Score Matching," Journal of Economic Surveys, Wiley Blackwell, vol. 22(1), pages 31-72, February.
    5. Sekhon, Jasjeet S., 2011. "Multivariate and Propensity Score Matching Software with Automated Balance Optimization: The Matching package for R," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 42(i07).
    6. Guido W. Imbens, 2004. "Nonparametric Estimation of Average Treatment Effects Under Exogeneity: A Review," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 4-29, February.
    7. Iacus, Stefano M. & King, Gary & Porro, Giuseppe, 2012. "Causal Inference without Balance Checking: Coarsened Exact Matching," Political Analysis, Cambridge University Press, vol. 20(1), pages 1-24, January.
    8. James J. Heckman & Hidehiko Ichimura & Petra Todd, 1998. "Matching As An Econometric Evaluation Estimator," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 65(2), pages 261-294.
    9. Imbens,Guido W. & Rubin,Donald B., 2015. "Causal Inference for Statistics, Social, and Biomedical Sciences," Cambridge Books, Cambridge University Press, number 9780521885881, November.
    10. Ho, Daniel E. & Imai, Kosuke & King, Gary & Stuart, Elizabeth A., 2007. "Matching as Nonparametric Preprocessing for Reducing Model Dependence in Parametric Causal Inference," Political Analysis, Cambridge University Press, vol. 15(3), pages 199-236, July.
    11. Zhong Zhao, 2004. "Using Matching to Estimate Treatment Effects: Data Requirements, Matching Metrics, and Monte Carlo Evidence," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 91-107, February.
    12. Rosenbaum, Paul R. & Ross, Richard N. & Silber, Jeffrey H., 2007. "Minimum Distance Matched Sampling With Fine Balance in an Observational Study of Treatment for Ovarian Cancer," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 75-83, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Eli Ben‐Michael & Avi Feller & Jesse Rothstein, 2022. "Synthetic controls with staggered adoption," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 84(2), pages 351-381, April.
    2. Adeola Oyenubi & Martin Wittenberg, 2021. "Does the choice of balance-measure matter under genetic matching?," Empirical Economics, Springer, vol. 61(1), pages 489-502, July.
    3. Gustavo Canavire-Bacarreza & Luis Castro Peñarrieta & Darwin Ugarte Ontiveros, 2021. "Outliers in Semi-Parametric Estimation of Treatment Effects," Econometrics, MDPI, vol. 9(2), pages 1-32, April.
    4. Aleksandra Kacperczyk & Chanchal Balachandran, 2018. "Vertical and Horizontal Wage Dispersion and Mobility Outcomes: Evidence from the Swedish Microdata," Organization Science, INFORMS, vol. 29(1), pages 17-38, February.
    5. Zhang, Chi & Managi, Shunsuke, 2020. "Functional social support and maternal stress: A study on the 2017 paid parental leave reform in Japan," Economic Analysis and Policy, Elsevier, vol. 65(C), pages 153-172.
    6. Christopher F. Baum & Hans Lööf & Andreas Stephan & Klaus F. Zimmermann, 2018. "Estimating the wage premia of refugee immigrants: Lessons from Sweden," Boston College Working Papers in Economics 963, Boston College Department of Economics, revised 22 Feb 2024.
    7. Per J. Agrell & Pontus Mattsson & Jonas Månsson, 2020. "Impacts on efficiency of merging the Swedish district courts," Annals of Operations Research, Springer, vol. 288(2), pages 653-679, May.
    8. F Baum, Christopher & Lööf, Hans & Stephan, Andreas & F. Zimmermann, Klaus, 2020. "Productivity of refugee workers and implications for innovation and growth," Working Paper Series in Economics and Institutions of Innovation 485, Royal Institute of Technology, CESIS - Centre of Excellence for Science and Innovation Studies, revised 24 Mar 2022.
    9. Allard Duursma, 2021. "Making disorder more manageable: The short-term effectiveness of local mediation in Darfur," Journal of Peace Research, Peace Research Institute Oslo, vol. 58(3), pages 554-567, May.
    10. Woo, Hongjoo & Chung, Angie & Richey, Robert Glenn & Hopkins, Christopher & Lee, Kangbok, 2022. "Product-flyer location and type of product categories in retailing," Journal of Business Research, Elsevier, vol. 138(C), pages 146-160.
    11. Ben Weidmann & Luke Miratrix, 2021. "Lurking Inferential Monsters? Quantifying Selection Bias In Evaluations Of School Programs," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 40(3), pages 964-986, June.
    12. Margaret E. Roberts & Brandon M. Stewart & Richard A. Nielsen, 2020. "Adjusting for Confounding with Text Matching," American Journal of Political Science, John Wiley & Sons, vol. 64(4), pages 887-903, October.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Dettmann, E. & Becker, C. & Schmeißer, C., 2011. "Distance functions for matching in small samples," Computational Statistics & Data Analysis, Elsevier, vol. 55(5), pages 1942-1960, May.
    2. Adeola Oyenubi & Martin Wittenberg, 2021. "Does the choice of balance-measure matter under genetic matching?," Empirical Economics, Springer, vol. 61(1), pages 489-502, July.
    3. Flores, Carlos A. & Mitnik, Oscar A., 2009. "Evaluating Nonexperimental Estimators for Multiple Treatments: Evidence from Experimental Data," IZA Discussion Papers 4451, Institute of Labor Economics (IZA).
    4. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    5. Timothy B. Armstrong & Michal Kolesár, 2021. "Finite‐Sample Optimal Estimation and Inference on Average Treatment Effects Under Unconfoundedness," Econometrica, Econometric Society, vol. 89(3), pages 1141-1177, May.
    6. Ferraro, Paul J. & Miranda, Juan José, 2014. "The performance of non-experimental designs in the evaluation of environmental programs: A design-replication study using a large-scale randomized experiment as a benchmark," Journal of Economic Behavior & Organization, Elsevier, vol. 107(PA), pages 344-365.
    7. Steven Lehrer & Gregory Kordas, 2013. "Matching using semiparametric propensity scores," Empirical Economics, Springer, vol. 44(1), pages 13-45, February.
    8. Jones A.M & Rice N, 2009. "Econometric Evaluation of Health Policies," Health, Econometrics and Data Group (HEDG) Working Papers 09/09, HEDG, c/o Department of Economics, University of York.
    9. Huber, Martin & Lechner, Michael & Wunsch, Conny, 2013. "The performance of estimators based on the propensity score," Journal of Econometrics, Elsevier, vol. 175(1), pages 1-21.
    10. Huber, Martin & Lechner, Michael & Wunsch, Conny, 2010. "How to Control for Many Covariates? Reliable Estimators Based on the Propensity Score," IZA Discussion Papers 5268, Institute of Labor Economics (IZA).
    11. Tommaso Nannicini, 2007. "Simulation-based sensitivity analysis for matching estimators," Stata Journal, StataCorp LP, vol. 7(3), pages 334-350, September.
    12. Dettmann, Eva & Becker, Claudia & Schmeißer, Christian, 2010. "Is there a Superior Distance Function for Matching in Small Samples?," IWH Discussion Papers 3/2010, Halle Institute for Economic Research (IWH).
    13. Jochen Kluve & Boris Augurzky, 2007. "Assessing the performance of matching algorithms when selection into treatment is strong," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 22(3), pages 533-557.
    14. Siying Guo & Jianxuan Liu & Qiu Wang, 2022. "Effective Learning During COVID-19: Multilevel Covariates Matching and Propensity Score Matching," Annals of Data Science, Springer, vol. 9(5), pages 967-982, October.
    15. Matthew A. COLE & Robert R.J. ELLIOTT & OKUBO Toshihiro & Liyun ZHANG, 2017. "The Pollution Outsourcing Hypothesis: An empirical test for Japan," Discussion papers 17096, Research Institute of Economy, Trade and Industry (RIETI).
    16. Robert J. R. Elliott & Liza Jabbour & Liyun Zhang, 2016. "Firm productivity and importing: Evidence from Chinese manufacturing firms," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 49(3), pages 1086-1124, August.
    17. Peter R. Mueser & Kenneth R. Troske & Alexey Gorislavsky, 2007. "Using State Administrative Data to Measure Program Performance," The Review of Economics and Statistics, MIT Press, vol. 89(4), pages 761-783, November.
    18. Wendimu, Mengistu Assefa & Henningsen, Arne & Gibbon, Peter, 2016. "Sugarcane Outgrowers in Ethiopia: “Forced” to Remain Poor?," World Development, Elsevier, vol. 83(C), pages 84-97.
    19. Gerhard Krug, 2017. "Augmenting propensity score equations to avoid misspecification bias – Evidence from a Monte Carlo simulation [Erweiterung der Propensity Score Gleichung zur Vermeidung von Fehlspezifikationen? Ein," AStA Wirtschafts- und Sozialstatistisches Archiv, Springer;Deutsche Statistische Gesellschaft - German Statistical Society, vol. 11(3), pages 205-231, December.
    20. Dudel Christian & Garbuszus Jan Marvin & Ott Notburga & Werding Martin, 2017. "Matching as Non-Parametric Preprocessing for the Estimation of Equivalence Scales," Journal of Economics and Statistics (Jahrbuecher fuer Nationaloekonomie und Statistik), De Gruyter, vol. 237(2), pages 115-141, April.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:amposc:v:61:y:2017:i:2:p:473-489. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://doi.org/10.1111/(ISSN)1540-5907 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.