IDEAS home Printed from https://ideas.repec.org/a/eee/soceco/v103y2023ics2214804323000186.html
   My bibliography  Save this article

Measuring non-cognitive skills exploiting log-files on online behaviour

Author

Listed:
  • Zhang, Yi
  • He, Jia

Abstract

Non-cognitive skills are important components of individuals’ human capital and good predictors of educational and labour market outcomes. Conventional self-reported measures of non-cognitive skills suffer from measurement errors stemming from self-presentation and reference group effects, which can produce paradoxical results in cross-country comparisons. We propose a novel source of measures derived from computer-generated log files on the behaviour of individuals taking an online test or respondents taking an online survey. We analyse measures of two desirable non-cognitive skills, perseverance and deep learning, constructed with log-file data from two large-scale educational surveys. Compared with the self-reported measures, our log-based behavioural measures have higher cross-country comparability, as they predict the performance of tests consistently at both individual and country levels. They also show high predictive validity in schooling and labour market outcomes, offering promise for a wide range of applications. We discuss the methodological implications of log-based behavioural measures and encourage researchers to apply them in combination with conventional self-reported measures.

Suggested Citation

  • Zhang, Yi & He, Jia, 2023. "Measuring non-cognitive skills exploiting log-files on online behaviour," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 103(C).
    • Handle: RePEc:eee:soceco:v:103:y:2023:i:c:s2214804323000186
    DOI: 10.1016/j.socec.2023.101992
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S2214804323000186
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.socec.2023.101992?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Arie Kapteyn & James P. Smith & Arthur van Soest, 2007. "Vignettes and Self-Reports of Work Disability in the United States and the Netherlands," American Economic Review, American Economic Association, vol. 97(1), pages 461-473, March.
    2. Yuanyuan Chen & Shuaizhang Feng & James J. Heckman & Tim Kautz, 2020. "Sensitivity of self-reported noncognitive skills to survey administration conditions," Proceedings of the National Academy of Sciences, Proceedings of the National Academy of Sciences, vol. 117(2), pages 931-935, January.
    3. James J. Heckman & Jora Stixrud & Sergio Urzua, 2006. "The Effects of Cognitive and Noncognitive Abilities on Labor Market Outcomes and Social Behavior," Journal of Labor Economics, University of Chicago Press, vol. 24(3), pages 411-482, July.
    4. Weiss, Andrew, 1988. "High School Graduation, Performance, and Wages," Journal of Political Economy, University of Chicago Press, vol. 96(4), pages 785-820, August.
    5. Trivitt, Julie & Cheng, Albert, 2016. "When you say nothing at all: The predictive power of student effort on surveysAuthor-Name: Hitt, Collin," Economics of Education Review, Elsevier, vol. 52(C), pages 105-119.
    6. Zamarro, Gema & Cheng, Albert & Shakeel, M. Danish & Hitt, Collin, 2018. "Comparing and validating measures of non-cognitive traits: Performance task measures and self-reports from a nationally representative internet panel," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 72(C), pages 51-60.
    7. Erik Lindqvist & Roine Vestman, 2011. "The Labor Market Returns to Cognitive and Noncognitive Ability: Evidence from the Swedish Enlistment," American Economic Journal: Applied Economics, American Economic Association, vol. 3(1), pages 101-128, January.
    8. Timothy N. Bond & Kevin Lang, 2019. "The Sad Truth about Happiness Scales," Journal of Political Economy, University of Chicago Press, vol. 127(4), pages 1629-1640.
    9. Hana Voňková & Patrick Hullegie, 2011. "Is the anchoring vignette method sensitive to the domain and choice of the vignette?," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 174(3), pages 597-620, July.
    10. Yona Rubinstein & James J. Heckman, 2001. "The Importance of Noncognitive Skills: Lessons from the GED Testing Program," American Economic Review, American Economic Association, vol. 91(2), pages 145-149, May.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Marianna Battaglia & Marisa Hidalgo-Hidalgo, 2020. "Non-Cognitive Skills and Remedial Education: Good News for Girls," Working Papers 20.10, Universidad Pablo de Olavide, Department of Economics.
    2. Marcenaro-Gutierrez, O.D. & Lopez-Agudo, L.A. & Henriques, C.O., 2021. "Are soft skills conditioned by conflicting factors? A multiobjective programming approach to explore the trade-offs," Economic Analysis and Policy, Elsevier, vol. 72(C), pages 18-40.
    3. Shelly Lundberg, 2017. "Noncognitive Skills as Human Capital," NBER Chapters, in: Education, Skills, and Technical Change: Implications for Future US GDP Growth, pages 219-243, National Bureau of Economic Research, Inc.
    4. Shelly Lundberg, 2013. "The College Type: Personality and Educational Inequality," Journal of Labor Economics, University of Chicago Press, vol. 31(3), pages 421-441.
    5. Lin, Dajun & Lutter, Randall & Ruhm, Christopher J., 2018. "Cognitive performance and labour market outcomes," Labour Economics, Elsevier, vol. 51(C), pages 121-135.
    6. Murphy, Richard & Weinhardt, Felix, 2020. "Top of the Class: The Importance of Ordinal Rank," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 87(6), pages 2777-2826.
    7. Gong, Xiaodong & Zhu, Rong, 2019. "Cognitive abilities, non-cognitive skills, and gambling behaviors," Journal of Economic Behavior & Organization, Elsevier, vol. 165(C), pages 51-69.
    8. Fuchs, Benjamin, 2015. "The Effect of Teenage Employment on Character Skills and Occupational Choice Strategies," VfS Annual Conference 2015 (Muenster): Economic Development - Theory and Policy 113030, Verein für Socialpolitik / German Economic Association.
    9. Haeckl, Simone & Kartal, Melis, 2021. "Does a stereotype benefit women in the labor market: An experiment on perseverance," UiS Working Papers in Economics and Finance 2021/5, University of Stavanger.
    10. Sandra E. Black & Erik Grönqvist & Björn Öckert, 2018. "Born to Lead? The Effect of Birth Order on Noncognitive Abilities," The Review of Economics and Statistics, MIT Press, vol. 100(2), pages 274-286, May.
    11. Helena Holmlund & Olmo Silva, 2014. "Targeting Noncognitive Skills to Improve Cognitive Outcomes: Evidence from a Remedial Education Intervention," Journal of Human Capital, University of Chicago Press, vol. 8(2), pages 126-160.
    12. Kamis, Rais & Pan, Jessica & Seah, Kelvin KC, 2023. "Do college admissions criteria matter? Evidence from discretionary vs. grade-based admission policies," Economics of Education Review, Elsevier, vol. 92(C).
    13. Balart, Pau & Oosterveen, Matthijs & Webbink, Dinand, 2018. "Test scores, noncognitive skills and economic growth," Economics of Education Review, Elsevier, vol. 63(C), pages 134-153.
    14. Mark Mitchell & Marta Favara & Catherine Porter & Alan Sánchez, 2020. "Human Capital Development," Working Papers 308280110, Lancaster University Management School, Economics Department.
    15. Bonacini, Luca & Gallo, Giovanni & Scicchitano, Sergio, 2021. "Sometimes you cannot make it on your own. How household background influences chances of success in Italy," GLO Discussion Paper Series 832, Global Labor Organization (GLO).
    16. Andrew Agopsowicz & Chris Robinson & Ralph Stinebrickner & Todd Stinebrickner, 2020. "Careers and Mismatch for College Graduates: College and Noncollege Jobs," Journal of Human Resources, University of Wisconsin Press, vol. 55(4), pages 1194-1221.
    17. Jose Maria Cabrera & Dinand Webbink, 2020. "Do Higher Salaries Yield Better Teachers and Better Student Outcomes?," Journal of Human Resources, University of Wisconsin Press, vol. 55(4), pages 1222-1257.
    18. Raj Chetty & John N. Friedman & Nathaniel Hilger & Emmanuel Saez & Diane Whitmore Schanzenbach & Danny Yagan, 2011. "How Does Your Kindergarten Classroom Affect Your Earnings? Evidence from Project Star," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 126(4), pages 1593-1660.
    19. Ahn, Taehyun & Goh, Young-Geun, 2021. "The long-term influences of ability mixing on soft skills," Journal of Economic Behavior & Organization, Elsevier, vol. 191(C), pages 367-386.
    20. Germán Reyes, 2023. "Cognitive Endurance, Talent Selection, and the Labor Market Returns to Human Capital," CRC TR 224 Discussion Paper Series crctr224_2023_490, University of Bonn and University of Mannheim, Germany.

    More about this item

    Keywords

    Non-cognitive skill; Self-reported measure; Behavioural measure; Log-file; Comparability; Predictive performance;
    All these keywords.

    JEL classification:

    • D91 - Microeconomics - - Micro-Based Behavioral Economics - - - Role and Effects of Psychological, Emotional, Social, and Cognitive Factors on Decision Making
    • I21 - Health, Education, and Welfare - - Education - - - Analysis of Education
    • J24 - Labor and Demographic Economics - - Demand and Supply of Labor - - - Human Capital; Skills; Occupational Choice; Labor Productivity
    • N30 - Economic History - - Labor and Consumers, Demography, Education, Health, Welfare, Income, Wealth, Religion, and Philanthropy - - - General, International, or Comparative

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:soceco:v:103:y:2023:i:c:s2214804323000186. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/inca/620175 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.