IDEAS home Printed from https://ideas.repec.org/a/taf/japsta/v41y2014i8p1767-1784.html
   My bibliography  Save this article

Sensitivity analysis of the unconfoundedness assumption with an application to an evaluation of college choice effects on earnings

Author

Listed:
  • Xavier de Luna
  • Mathias Lundin

Abstract

We evaluate the effects of college choice on earnings using Swedish register databases. This case study is used to motivate the introduction of a novel procedure to analyse the sensitivity of such an observational study to the assumption made that there are no unobserved confounders - variables affecting both college choice and earnings. This assumption is not testable without further information, and should be considered an approximation of reality. To perform a sensitivity analysis, we measure the departure from the unconfoundedness assumption with the correlation between college choice and earnings when conditioning on observed covariates. The use of a correlation as a measure of dependence allows us to propose a standardised procedure by advocating the use of a fixed value for the correlation, typically 1% or 5%, when checking the sensitivity of an evaluation study. A correlation coefficient is, moreover, intuitive to most empirical scientists, which makes the results of our sensitivity analysis easier to communicate than those of previously proposed methods. In our evaluation of the effects of college choice on earnings, the significantly positive effect obtained could not be questioned by a sensitivity analysis allowing for unobserved confounders inducing at most 5% correlation between college choice and earnings.

Suggested Citation

  • Xavier de Luna & Mathias Lundin, 2014. "Sensitivity analysis of the unconfoundedness assumption with an application to an evaluation of college choice effects on earnings," Journal of Applied Statistics, Taylor & Francis Journals, vol. 41(8), pages 1767-1784, August.
    • Handle: RePEc:taf:japsta:v:41:y:2014:i:8:p:1767-1784
    DOI: 10.1080/02664763.2014.890178
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1080/02664763.2014.890178
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1080/02664763.2014.890178?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Joseph G. Altonji & Todd E. Elder & Christopher R. Taber, 2005. "Selection on Observed and Unobserved Variables: Assessing the Effectiveness of Catholic Schools," Journal of Political Economy, University of Chicago Press, vol. 113(1), pages 151-184, February.
    2. Andrea Ichino & Fabrizia Mealli & Tommaso Nannicini, 2008. "From temporary help jobs to permanent employment: what can we learn from matching estimators and their sensitivity?," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 23(3), pages 305-327.
    3. Guildo W. Imbens, 2003. "Sensitivity to Exogeneity Assumptions in Program Evaluation," American Economic Review, American Economic Association, vol. 93(2), pages 126-132, May.
    4. Long, Mark C., 2010. "Changes in the returns to education and college quality," Economics of Education Review, Elsevier, vol. 29(3), pages 338-347, June.
    5. DiPrete, Thomas A. & Gangl, Markus, 2004. "Assessing bias in the estimation of causal effects: Rosenbaum bounds on matching estimators and instrumental variables estimation with imperfect instruments," Discussion Papers, Research Unit: Labor Market Policy and Employment SP I 2004-101, WZB Berlin Social Science Center.
    6. de Luna, Xavier & Johansson, Per, 2006. "Exogeneity in structural equation models," Journal of Econometrics, Elsevier, vol. 132(2), pages 527-543, June.
    7. John Copas & Shinto Eguchi, 2001. "Local sensitivity approximations for selectivity bias," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 63(4), pages 871-895.
    8. Stacy Berg Dale & Alan B. Krueger, 2002. "Estimating the Payoff to Attending a More Selective College: An Application of Selection on Observables and Unobservables," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 117(4), pages 1491-1527.
    9. Heckman, James, 2013. "Sample selection bias as a specification error," Applied Econometrics, Russian Presidential Academy of National Economy and Public Administration (RANEPA), vol. 31(3), pages 129-137.
    10. Sander Greenland, 2005. "Multiple‐bias modelling for analysis of observational data," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 168(2), pages 267-306, March.
    11. Black, Dan A. & Smith, J.A.Jeffrey A., 2004. "How robust is the evidence on the effects of college quality? Evidence from matching," Journal of Econometrics, Elsevier, vol. 121(1-2), pages 99-124.
    12. Simon N. Wood, 2003. "Thin plate regression splines," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 65(1), pages 95-114, February.
    13. Little, Roderick J A, 1985. "A Note about Models for Selectivity Bias," Econometrica, Econometric Society, vol. 53(6), pages 1469-1474, November.
    14. Lena Lindahl & Håkan Regnér, 2005. "College Choice and Subsequent Earnings: Results Using Swedish Sibling Data," Scandinavian Journal of Economics, Wiley Blackwell, vol. 107(3), pages 437-457, September.
    15. John Copas & Shinto Eguchi, 2005. "Local model uncertainty and incomplete‐data bias (with discussion)," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(4), pages 459-513, September.
    16. Dominic J. Brewer & Eric R. Eide & Ronald G. Ehrenberg, 1999. "Does It Pay to Attend an Elite Private College? Cross-Cohort Evidence on the Effects of College Type on Earnings," Journal of Human Resources, University of Wisconsin Press, vol. 34(1), pages 104-123.
    17. Eliasson, Kent, 2006. "College Choice And Earnings Among University Graduates In Sweden," Umeå Economic Studies 693, Umeå University, Department of Economics.
    18. Mark Hoekstra, 2009. "The Effect of Attending the Flagship State University on Earnings: A Discontinuity-Based Approach," The Review of Economics and Statistics, MIT Press, vol. 91(4), pages 717-724, November.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Ji, Sisi & Zhu, Zheyi, 2022. "Does higher education matter for health?," Cardiff Economics Working Papers E2022/4, Cardiff University, Cardiff Business School, Economics Section.
    2. Bacalhau, Priscilla & Mattos, Enlinson & Ponczek, Vladimir Pinheiro, 2019. "College quality signaling and individual performance: effects on labor market outcomes after graduation," Textos para discussão 502, FGV EESP - Escola de Economia de São Paulo, Fundação Getulio Vargas (Brazil).
    3. Arnaud Chevalier, 2014. "Does Higher Education Quality Matter in the UK?," Research in Labor Economics, in: Factors Affecting Worker Well-being: The Impact of Change in the Labor Market, volume 40, pages 257-292, Emerald Group Publishing Limited.
    4. Minna Genbäck & Elena Stanghellini & Xavier Luna, 2015. "Uncertainty intervals for regression parameters with non-ignorable missingness in the outcome," Statistical Papers, Springer, vol. 56(3), pages 829-847, August.
    5. Jenny Häggström & Xavier Luna, 2014. "Targeted smoothing parameter selection for estimating average causal effects," Computational Statistics, Springer, vol. 29(6), pages 1727-1748, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Suhonen, Tuomo, 2013. "Are there returns from university location in a state-funded university system?," Regional Science and Urban Economics, Elsevier, vol. 43(3), pages 465-478.
    2. Eliasson, Kent, 2006. "The Role of Ability in Estimating the Returns to College Choice: New Swedish Evidence," Umeå Economic Studies 691, Umeå University, Department of Economics.
    3. Zara Daghbashyan & Björn Hårsman, 2014. "University choice and entrepreneurship," Small Business Economics, Springer, vol. 42(4), pages 729-746, April.
    4. Justine S. Hastings & Christopher A. Neilson & Seth D. Zimmerman, 2013. "Are Some Degrees Worth More than Others? Evidence from college admission cutoffs in Chile," NBER Working Papers 19241, National Bureau of Economic Research, Inc.
    5. Broecke, Stijn, 2012. "University selectivity and earnings: Evidence from UK data on applications and admissions to university," Economics of Education Review, Elsevier, vol. 31(3), pages 96-107.
    6. Smith, Jonathan, 2013. "Ova and out: Using twins to estimate the educational returns to attending a selective college," Economics of Education Review, Elsevier, vol. 36(C), pages 166-180.
    7. Seki, Mai, 2014. "Heterogeneous Returns to U.S. College Selectivity and the Value of Graduate Degree Attainment," CLSSRN working papers clsrn_admin-2014-53, Vancouver School of Economics, revised 25 Nov 2014.
    8. Peter Arcidiacono & Michael Lovenheim, 2016. "Affirmative Action and the Quality-Fit Trade-Off," Journal of Economic Literature, American Economic Association, vol. 54(1), pages 3-51, March.
    9. Li, Hongbin & Meng, Lingsheng & Shi, Xinzheng & Wu, Binzhen, 2012. "Does attending elite colleges pay in China?," Journal of Comparative Economics, Elsevier, vol. 40(1), pages 78-88.
    10. Hinrichs, Peter, 2014. "Affirmative action bans and college graduation rates," Economics of Education Review, Elsevier, vol. 42(C), pages 43-52.
    11. Andrews, Rodney & Li, Jing & Lovenheim, Michael F., 2014. "Heterogeneous paths through college: Detailed patterns and relationships with graduation and earnings," Economics of Education Review, Elsevier, vol. 42(C), pages 93-108.
    12. Milla, Joniada, 2017. "The Context-Bound University Selectivity Premium," IZA Discussion Papers 11025, Institute of Labor Economics (IZA).
    13. Weinstein, Russell, 2018. "Employer screening costs, recruiting strategies, and labor market outcomes: An equilibrium analysis of on-campus recruiting," Labour Economics, Elsevier, vol. 55(C), pages 282-299.
    14. Leah Achdut & Elad Gutman & Idan Lipiner & Inbal Maayan & Noam Zussman, 2018. "The Wage Premium on Higher Education: Universities and Colleges," Bank of Israel Working Papers 2018.13, Bank of Israel.
    15. Tuomo Suhonen, 2014. "Quality of higher education and earnings: evidence from Finland using field-of-study-level quality measures," International Review of Applied Economics, Taylor & Francis Journals, vol. 28(1), pages 22-44, January.
    16. Long, Mark C., 2010. "Changes in the returns to education and college quality," Economics of Education Review, Elsevier, vol. 29(3), pages 338-347, June.
    17. Kirkebøen, Lars & Leuven, Edwin & Mogstad, Magne, 2014. "Field of Study, Earnings, and Self-Selection," Memorandum 29/2014, Oslo University, Department of Economics.
    18. Drydakis, Nick, 2016. "The effect of university attended on graduates’ labour market prospects: A field study of Great Britain," Economics of Education Review, Elsevier, vol. 52(C), pages 192-208.
    19. Gartell, Marie, 2009. "Stability of college rankings - A study of relative earnings estimates applying different methods and models on Swedish data," Working Paper Series 2009:19, IFAU - Institute for Evaluation of Labour Market and Education Policy.
    20. Stacy Dale & Alan B. Krueger, "undated". "Estimating the Return to College Selectivity Over the Career Using Administrative Earning Data," Mathematica Policy Research Reports d76ec29a0bbb4b1bb9d285b5a, Mathematica Policy Research.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:taf:japsta:v:41:y:2014:i:8:p:1767-1784. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Longhurst (email available below). General contact details of provider: http://www.tandfonline.com/CJAS20 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.