IDEAS home Printed from https://ideas.repec.org/a/eee/soceco/v103y2023ics2214804323000186.html
   My bibliography  Save this article

Measuring non-cognitive skills exploiting log-files on online behaviour

Author

Listed:
  • Zhang, Yi
  • He, Jia

Abstract

Non-cognitive skills are important components of individuals’ human capital and good predictors of educational and labour market outcomes. Conventional self-reported measures of non-cognitive skills suffer from measurement errors stemming from self-presentation and reference group effects, which can produce paradoxical results in cross-country comparisons. We propose a novel source of measures derived from computer-generated log files on the behaviour of individuals taking an online test or respondents taking an online survey. We analyse measures of two desirable non-cognitive skills, perseverance and deep learning, constructed with log-file data from two large-scale educational surveys. Compared with the self-reported measures, our log-based behavioural measures have higher cross-country comparability, as they predict the performance of tests consistently at both individual and country levels. They also show high predictive validity in schooling and labour market outcomes, offering promise for a wide range of applications. We discuss the methodological implications of log-based behavioural measures and encourage researchers to apply them in combination with conventional self-reported measures.

Suggested Citation

  • Zhang, Yi & He, Jia, 2023. "Measuring non-cognitive skills exploiting log-files on online behaviour," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 103(C).
    • Handle: RePEc:eee:soceco:v:103:y:2023:i:c:s2214804323000186
    DOI: 10.1016/j.socec.2023.101992
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S2214804323000186
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.socec.2023.101992?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Hana Voňková & Patrick Hullegie, 2011. "Is the anchoring vignette method sensitive to the domain and choice of the vignette?," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 174(3), pages 597-620, July.
    2. Arie Kapteyn & James P. Smith & Arthur van Soest, 2007. "Vignettes and Self-Reports of Work Disability in the United States and the Netherlands," American Economic Review, American Economic Association, vol. 97(1), pages 461-473, March.
    3. Timothy N. Bond & Kevin Lang, 2019. "The Sad Truth about Happiness Scales," Journal of Political Economy, University of Chicago Press, vol. 127(4), pages 1629-1640.
    4. Zamarro, Gema & Cheng, Albert & Shakeel, M. Danish & Hitt, Collin, 2018. "Comparing and validating measures of non-cognitive traits: Performance task measures and self-reports from a nationally representative internet panel," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 72(C), pages 51-60.
    5. Yuanyuan Chen & Shuaizhang Feng & James J. Heckman & Tim Kautz, 2020. "Sensitivity of self-reported noncognitive skills to survey administration conditions," Proceedings of the National Academy of Sciences, Proceedings of the National Academy of Sciences, vol. 117(2), pages 931-935, January.
    6. James J. Heckman & Jora Stixrud & Sergio Urzua, 2006. "The Effects of Cognitive and Noncognitive Abilities on Labor Market Outcomes and Social Behavior," Journal of Labor Economics, University of Chicago Press, vol. 24(3), pages 411-482, July.
    7. Erik Lindqvist & Roine Vestman, 2011. "The Labor Market Returns to Cognitive and Noncognitive Ability: Evidence from the Swedish Enlistment," American Economic Journal: Applied Economics, American Economic Association, vol. 3(1), pages 101-128, January.
    8. Trivitt, Julie & Cheng, Albert, 2016. "When you say nothing at all: The predictive power of student effort on surveysAuthor-Name: Hitt, Collin," Economics of Education Review, Elsevier, vol. 52(C), pages 105-119.
    9. Weiss, Andrew, 1988. "High School Graduation, Performance, and Wages," Journal of Political Economy, University of Chicago Press, vol. 96(4), pages 785-820, August.
    10. Yona Rubinstein & James J. Heckman, 2001. "The Importance of Noncognitive Skills: Lessons from the GED Testing Program," American Economic Review, American Economic Association, vol. 91(2), pages 145-149, May.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Marianna Battaglia & Marisa Hidalgo-Hidalgo, 2020. "Non-Cognitive Skills and Remedial Education: Good News for Girls," Working Papers 20.10, Universidad Pablo de Olavide, Department of Economics.
    2. Marcenaro-Gutierrez, O.D. & Lopez-Agudo, L.A. & Henriques, C.O., 2021. "Are soft skills conditioned by conflicting factors? A multiobjective programming approach to explore the trade-offs," Economic Analysis and Policy, Elsevier, vol. 72(C), pages 18-40.
    3. Shelly Lundberg, 2017. "Noncognitive Skills as Human Capital," NBER Chapters, in: Education, Skills, and Technical Change: Implications for Future US GDP Growth, pages 219-243, National Bureau of Economic Research, Inc.
    4. Lin, Dajun & Lutter, Randall & Ruhm, Christopher J., 2018. "Cognitive performance and labour market outcomes," Labour Economics, Elsevier, vol. 51(C), pages 121-135.
    5. Alain Cohn & Michel André Maréchal & Frédéric Schneider & Roberto A Weber, 2021. "Frequent Job Changes can Signal Poor Work Attitude and Reduce Employability," Journal of the European Economic Association, European Economic Association, vol. 19(1), pages 475-508.
    6. Shelly Lundberg, 2013. "The College Type: Personality and Educational Inequality," Journal of Labor Economics, University of Chicago Press, vol. 31(3), pages 421-441.
    7. Raj Chetty & John N. Friedman & Nathaniel Hilger & Emmanuel Saez & Diane Whitmore Schanzenbach & Danny Yagan, 2011. "How Does Your Kindergarten Classroom Affect Your Earnings? Evidence from Project Star," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 126(4), pages 1593-1660.
    8. Petter Lundborg & Paul Nystedt & Dan-Olof Rooth, 2014. "Body Size, Skills, and Income: Evidence From 150,000 Teenage Siblings," Demography, Springer;Population Association of America (PAA), vol. 51(5), pages 1573-1596, October.
    9. C. Kirabo Jackson, 2012. "Non-Cognitive Ability, Test Scores, and Teacher Quality: Evidence from 9th Grade Teachers in North Carolina," NBER Working Papers 18624, National Bureau of Economic Research, Inc.
    10. Richard Murphy & Felix Weinhardt, 2020. "Top of the Class: The Importance of Ordinal Rank," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 87(6), pages 2777-2826.
    11. Anna Bussu & Dimitri Paolini & Manuela Pulina & Giuseppe Zanzurino, 2023. "From Choice to Performance in Secondary Schools: Evidence from a Disadvantaged Setting in Italy," Italian Economic Journal: A Continuation of Rivista Italiana degli Economisti and Giornale degli Economisti, Springer;Società Italiana degli Economisti (Italian Economic Association), vol. 9(2), pages 529-555, July.
    12. Sandra E. Black & Erik Grönqvist & Björn Öckert, 2018. "Born to Lead? The Effect of Birth Order on Noncognitive Abilities," The Review of Economics and Statistics, MIT Press, vol. 100(2), pages 274-286, May.
    13. Gong, Xiaodong & Zhu, Rong, 2019. "Cognitive abilities, non-cognitive skills, and gambling behaviors," Journal of Economic Behavior & Organization, Elsevier, vol. 165(C), pages 51-69.
    14. Melinda Petre, 2018. "Are Employers Omniscient? Employer Learning About Cognitive and Noncognitive Skills," Industrial Relations: A Journal of Economy and Society, Wiley Blackwell, vol. 57(3), pages 323-360, July.
    15. Fuchs, Benjamin, 2015. "The Effect of Teenage Employment on Character Skills and Occupational Choice Strategies," VfS Annual Conference 2015 (Muenster): Economic Development - Theory and Policy 113030, Verein für Socialpolitik / German Economic Association.
    16. Haeckl, Simone & Kartal, Melis, 2021. "Does a stereotype benefit women in the labor market: An experiment on perseverance," UiS Working Papers in Economics and Finance 2021/5, University of Stavanger.
    17. Sahn, David E. & Villa, Kira M., 2015. "The Role of Personality, Cognition and Shocks in Determining Age of Entry into Labor Market, Sector of Employment, and within Sector Earnings," 2015 AAEA & WAEA Joint Annual Meeting, July 26-28, San Francisco, California 205673, Agricultural and Applied Economics Association.
    18. Dinand Webbink & José María Cabrera, 2016. "Do higher salaries yield better teachers and better student outcomes?," Documentos de Trabajo/Working Papers 1604, Facultad de Ciencias Empresariales y Economia. Universidad de Montevideo..
    19. Elke Lüdemann, 2011. "Schooling and the Formation of Cognitive and Non-cognitive Outcomes," ifo Beiträge zur Wirtschaftsforschung, ifo Institute - Leibniz Institute for Economic Research at the University of Munich, number 39.
    20. Deng, Lanfang & Tong, Tingting, 2020. "Parenting style and the development of noncognitive ability in children," China Economic Review, Elsevier, vol. 62(C).

    More about this item

    Keywords

    Non-cognitive skill; Self-reported measure; Behavioural measure; Log-file; Comparability; Predictive performance;
    All these keywords.

    JEL classification:

    • D91 - Microeconomics - - Micro-Based Behavioral Economics - - - Role and Effects of Psychological, Emotional, Social, and Cognitive Factors on Decision Making
    • I21 - Health, Education, and Welfare - - Education - - - Analysis of Education
    • J24 - Labor and Demographic Economics - - Demand and Supply of Labor - - - Human Capital; Skills; Occupational Choice; Labor Productivity
    • N30 - Economic History - - Labor and Consumers, Demography, Education, Health, Welfare, Income, Wealth, Religion, and Philanthropy - - - General, International, or Comparative

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:soceco:v:103:y:2023:i:c:s2214804323000186. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/inca/620175 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.