IDEAS home Printed from https://ideas.repec.org/a/bla/scandj/v104y2002i1p147-171.html
   My bibliography  Save this article

Evaluating Social Policy by Experimental and Nonexperimental Methods

Author

Listed:
  • Espen Bratberg
  • Astrid Grasdal
  • Alf Erling Risa

Abstract

Although it is important to establish causal relationships in social policy evaluation, the effects are difficult to observe due to sample selection. To evaluate the performance of estimators designed to handle sample selection bias, we analyse data from a Norwegian rehabilitation project with a randomised experimental design. The data permit us to compare the performance of different nonexperimental estimators with the experimental results. In our case study we find that nonexperimental evaluation based on sample selection estimators with selection terms that fail to meet conventional levels of statistical significance is highly unreliable. The difference in difference estimator and propensity score matching estimators perform better in our context. JEL classification: C51; J24; H43; I12

Suggested Citation

  • Espen Bratberg & Astrid Grasdal & Alf Erling Risa, 2002. "Evaluating Social Policy by Experimental and Nonexperimental Methods," Scandinavian Journal of Economics, Wiley Blackwell, vol. 104(1), pages 147-171, March.
    • Handle: RePEc:bla:scandj:v:104:y:2002:i:1:p:147-171
    DOI: 10.1111/1467-9442.00276
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/1467-9442.00276
    Download Restriction: no
    File URL: https://libkey.io/10.1111/1467-9442.00276?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Other versions of this item:

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Astrid Grasdal, 2001. "The performance of sample selection estimators to control for attrition bias," Health Economics, John Wiley & Sons, Ltd., vol. 10(5), pages 385-398, July.
    2. Steven Glazerman & Dan M. Levy & David Myers, 2003. "Nonexperimental Versus Experimental Estimates of Earnings Impacts," The ANNALS of the American Academy of Political and Social Science, , vol. 589(1), pages 63-93, September.
    3. Fredriksson, Per G. & Millimet, D.L.Daniel L., 2004. "Comparative politics and environmental taxation," Journal of Environmental Economics and Management, Elsevier, vol. 48(1), pages 705-722, July.
    4. Jacek Liwiński, 2017. "Premia płacowa z kształcenia na studiach podyplomowych," Gospodarka Narodowa. The Polish Journal of Economics, Warsaw School of Economics, issue 5, pages 105-127.
    5. Hoken, Hisatoshi & Su, Qun, 2015. "Measuring the effect of agricultural cooperatives on household income using PSM-DID : a case study of a rice-producing cooperative in China," IDE Discussion Papers 539, Institute of Developing Economies, Japan External Trade Organization(JETRO).
    6. Hasan Bakhshi & John Edwards & Stephen Roper & Judy Scully & Duncan Shaw & Lorraine Morley & Nicola Rathbone, 2013. "An Experimental Approach to Industrial Policy Evaluation: The case of Creative Credits," Research Papers 0004, Enterprise Research Centre.
    7. List, John A. & McHone, W. Warren & Millimet, Daniel L., 2004. "Effects of environmental regulation on foreign and domestic plant births: is there a home field advantage?," Journal of Urban Economics, Elsevier, vol. 56(2), pages 303-326, September.
    8. Steven Stern & John Pepper & David Dean & Robert Schmidt, 2011. "The Effects of Vocational Rehabilitation for People with Mental Illlness," Virginia Economics Online Papers 382, University of Virginia, Department of Economics.
    9. Maasoumi, Esfandiar & Millimet, Daniel & Sarkar, Dipanwita, 2005. "The Distribution of Returns to Marriage," Departmental Working Papers 0503, Southern Methodist University, Department of Economics.
    10. Michalek, Jerzy & Ciaian, Pavel & Pokrivcak, Jan, 2018. "The impact of producer organizations on farm performance: The case study of large farms from Slovakia☆," Food Policy, Elsevier, vol. 75(C), pages 80-92.
    11. Slottje, Daniel J. & Millimet, Daniel L. & Buchanan, Michael J., 2007. "Econometric analysis of copyrights," Journal of Econometrics, Elsevier, vol. 139(2), pages 303-317, August.
    12. Katherine Baicker & Theodore Svoronos, 2019. "Testing the Validity of the Single Interrupted Time Series Design," CID Working Papers 364, Center for International Development at Harvard University.
    13. Pokrivcak, J. & Michalek, J. & Ciaian, P., 2018. "The impact of producer organisations on farm performance: A case study of large farms in Slovakia," 2018 Conference, July 28-August 2, 2018, Vancouver, British Columbia 277485, International Association of Agricultural Economists.
    14. Geoff Perry & Tim Maloney, 2008. "Economic Evaluation of the Training Opportunities Programme in New Zealand," Australian Journal of Labour Economics (AJLE), Bankwest Curtin Economics Centre (BCEC), Curtin Business School, vol. 11(2), pages 163-185.
    15. List John A. & Millimet Daniel L & McHone Warren, 2004. "The Unintended Disincentive in the Clean Air Act," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 4(2), pages 1-28, February.
    16. Katherine Baicker & Theodore Svoronos, 2019. "Testing the Validity of the Single Interrupted Time Series Design," NBER Working Papers 26080, National Bureau of Economic Research, Inc.
    17. Kai Zhao & Wanshu Wu & Junmei Ye, 2023. "The impact of “Mass Entrepreneurship and Innovation” policy on SMEs’ innovation: Using quasi‐natural experiments," Bulletin of Economic Research, Wiley Blackwell, vol. 75(2), pages 348-365, April.
    18. Hoken, Hisatoshi, 2016. "Participation in farmer's cooperatives and its effects on agricultural incomes : evidence from vegetable-producing areas in China," IDE Discussion Papers 578, Institute of Developing Economies, Japan External Trade Organization(JETRO).
    19. Takasaki, Yoshito, 2024. "Impacts of vocational training for persons with disabilities: Experimental evidence from Cambodia," Journal of Development Economics, Elsevier, vol. 169(C).
    20. Vivian C. Wong & Peter M. Steiner & Kylie L. Anglin, 2018. "What Can Be Learned From Empirical Evaluations of Nonexperimental Methods?," Evaluation Review, , vol. 42(2), pages 147-175, April.
    21. Bakhshi, Hasan & Edwards, John S. & Roper, Stephen & Scully, Judy & Shaw, Duncan & Morley, Lorraine & Rathbone, Nicola, 2015. "Assessing an experimental approach to industrial policy evaluation: Applying RCT+ to the case of Creative Credits," Research Policy, Elsevier, vol. 44(8), pages 1462-1472.

    More about this item

    JEL classification:

    • H30 - Public Economics - - Fiscal Policies and Behavior of Economic Agents - - - General
    • C44 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods: Special Topics - - - Operations Research; Statistical Decision Theory
    • C82 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Methodology for Collecting, Estimating, and Organizing Macroeconomic Data; Data Access

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:scandj:v:104:y:2002:i:1:p:147-171. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://onlinelibrary.wiley.com/journal/10.1111/(ISSN)1467-9442 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.