IDEAS home Printed from https://ideas.repec.org/p/hes/wpaper/0255.html
   My bibliography  Save this paper

Breaking the HISCO Barrier: Automatic Occupational Standardization with OccCANINE

Author

Listed:
  • Christian Møller Dahl

    (University of Southern Denmark)

  • Torben Johansen

    (University of Southern Denmark)

  • Christian Vedel

    (University of Southern Denmark)

Abstract

This paper introduces a new tool, OccCANINE, to automatically transform occupational descriptions into the HISCO classification system. The manual work involved in processing and classifying occupational descriptions is error-prone, tedious, and time-consuming. We finetune a preexisting language model (CANINE) to do this automatically, thereby performing in seconds and minutes what previously took days and weeks. The model is trained on 14 million pairs of occupational descriptions and HISCO codes in 13 different languages contributed by 22 different sources. Our approach is shown to have accuracy, recall, and precision above 90 percent. Our tool breaks the metaphorical HISCO barrier and makes this data readily available for analysis of occupational structures with broad applicability in economics, economic history, and various related disciplines.

Suggested Citation

  • Christian Møller Dahl & Torben Johansen & Christian Vedel, 2024. "Breaking the HISCO Barrier: Automatic Occupational Standardization with OccCANINE," Working Papers 0255, European Historical Economics Society (EHES).
    • Handle: RePEc:hes:wpaper:0255
    as

    Download full text from publisher

    File URL: https://www.ehes.org/wp/EHES_255.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Allen,Robert C., 2009. "The British Industrial Revolution in Global Perspective," Cambridge Books, Cambridge University Press, number 9780521868273, January.
    2. Berger, Thor, 2019. "Railroads and Rural Industrialization: evidence from a Historical Policy Experiment," Explorations in Economic History, Elsevier, vol. 74(C).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jeanet Sinding Bentzen & Nina Boberg-Fazlić & Paul Sharp & Christian Volmar Skovsgaard & Christian Vedel, 2024. "Assimilate for God: The Impact of Religious Divisions on Danish American Communities," Working Papers 0253, European Historical Economics Society (EHES).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Haris Kitsikopoulos, 2024. "Steam power diffusion in the British cotton and woolen industries, 1774–1800: the role of firm size," Review of Evolutionary Political Economy, Springer, vol. 5(3), pages 559-580, October.
    2. Thomas Baudin & Robert Stelter, 2022. "The rural exodus and the rise of Europe," Journal of Economic Growth, Springer, vol. 27(3), pages 365-414, September.
    3. Ferreira, Pedro Cavalcanti & Pessôa, Samuel & dos Santos, Marcelo Rodrigues, 2016. "Globalization And The Industrial Revolution," Macroeconomic Dynamics, Cambridge University Press, vol. 20(3), pages 643-666, April.
    4. Klas Rönnbäck, 2014. "Slave ownership and fossil fuel usage: a commentary," Climatic Change, Springer, vol. 122(1), pages 1-9, January.
    5. Allen, Robert C., 2012. "Backward into the future: The shift to coal and implications for the next energy transition," Energy Policy, Elsevier, vol. 50(C), pages 17-23.
    6. Vincent Geloso & Peter Lindert, 2020. "Relative costs of living, for richer and poorer, 1688–1914," Cliometrica, Springer;Cliometric Society (Association Francaise de Cliométrie), vol. 14(3), pages 417-442, September.
    7. Edwyna Harris & Sumner La Croix, 2019. "Prices, Wages, and Welfare in Early Colonial South Australia, 1836-1850," Monash Economics Working Papers 07-19, Monash University, Department of Economics.
    8. Broadberry, Stephen & Ghosal, Sayantan & Proto, Eugenio, 2011. "Is Anonymity the Missing Link Between Commercial and Industrial Revolution?," The Warwick Economics Research Paper Series (TWERPS) 974, University of Warwick, Department of Economics.
    9. Jane Humphries & Jacob Weisdorf, 2019. "Unreal Wages? Real Income and Economic Growth in England, 1260–1850," The Economic Journal, Royal Economic Society, vol. 129(623), pages 2867-2887.
    10. Nuno Palma & André C. Silva, 2024. "Spending A Windfall," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 65(1), pages 283-313, February.
    11. Robert C. Allen, 2015. "The high wage economy and the industrial revolution: a restatement," Economic History Review, Economic History Society, vol. 68(1), pages 1-22, February.
    12. Meissner, Christopher M., 2014. "Growth from Globalization? A View from the Very Long Run," Handbook of Economic Growth, in: Philippe Aghion & Steven Durlauf (ed.), Handbook of Economic Growth, edition 1, volume 2, chapter 8, pages 1033-1069, Elsevier.
    13. Taalbi, Josef, 2017. "What drives innovation? Evidence from economic history," Research Policy, Elsevier, vol. 46(8), pages 1437-1453.
    14. Jedwab, Remi & Vollrath, Dietrich, 2015. "Urbanization without growth in historical perspective," Explorations in Economic History, Elsevier, vol. 58(C), pages 1-21.
    15. Daniele Tavani & Luca Zamparelli, 2020. "Labor-Augmenting Technical Change and the Labor Share: New Microeconomic Foundations," Working Papers 2/20, Sapienza University of Rome, DISS.
    16. Maria Mwaipopo Fibaek, 2021. "Working Poor? A Study of Rural Workers' Economic Welfare in Kenya," Journal of International Development, John Wiley & Sons, Ltd., vol. 33(1), pages 41-69, January.
    17. James Roumasset, 2020. "Clubs, Coase, and the role of government," Philippine Review of Economics, University of the Philippines School of Economics and Philippine Economic Society, vol. 57(1), pages 1-17, June.
    18. Thomas Barnebeck Andersen & Jeanet Bentzen & Carl-Johan Dalgaard & Paul Sharp, 2010. "Religious Orders and Growth through Cultural Change in Pre-Industrial England," DEGIT Conference Papers c015_036, DEGIT, Dynamics, Economic Growth, and International Trade.
    19. Peretto, Pietro F., 2015. "From Smith to Schumpeter: A theory of take-off and convergence to sustained growth," European Economic Review, Elsevier, vol. 78(C), pages 1-26.
    20. Ken Tabata, 2013. "The Expansion of the Commercial Sector and the Child Quantity-Quality Transition in a Malthusian World," Discussion Paper Series 105, School of Economics, Kwansei Gakuin University, revised May 2013.

    More about this item

    Keywords

    Occupational Standardization; HISCO Classification System; Machine Learning in Economic History; Language Models;
    All these keywords.

    JEL classification:

    • C55 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Large Data Sets: Modeling and Analysis
    • C81 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Methodology for Collecting, Estimating, and Organizing Microeconomic Data; Data Access
    • J1 - Labor and Demographic Economics - - Demographic Economics
    • N01 - Economic History - - General - - - Development of the Discipline: Historiographical; Sources and Methods
    • N3 - Economic History - - Labor and Consumers, Demography, Education, Health, Welfare, Income, Wealth, Religion, and Philanthropy
    • N6 - Economic History - - Manufacturing and Construction
    • O33 - Economic Development, Innovation, Technological Change, and Growth - - Innovation; Research and Development; Technological Change; Intellectual Property Rights - - - Technological Change: Choices and Consequences; Diffusion Processes

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:hes:wpaper:0255. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Paul Sharp (email available below). General contact details of provider: https://edirc.repec.org/data/ehessea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.