IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2103.06017.html
   My bibliography  Save this paper

Relatedness in the Era of Machine Learning

Author

Listed:
  • Andrea Tacchella
  • Andrea Zaccaria
  • Marco Miccheli
  • Luciano Pietronero

Abstract

Relatedness is a quantification of how much two human activities are similar in terms of the inputs and contexts needed for their development. Under the idea that it is easier to move between related activities than towards unrelated ones, empirical approaches to quantify relatedness are currently used as predictive tools to inform policies and development strategies in governments, international organizations, and firms. Here we focus on countries' industries and we show that the standard, widespread approach of estimating Relatedness through the co-location of activities (e.g. Product Space) generates a measure of relatedness that performs worse than trivial auto-correlation prediction strategies. We argue that this is a consequence of the poor signal-to-noise ratio present in international trade data. In this paper we show two main findings. First, we find that a shift from two-products correlations (network-density based) to many-products correlations (decision trees) can dramatically improve the quality of forecasts with a corresponding reduction of the risk of wrong policy choices. Then, we propose a new methodology to empirically estimate Relatedness that we call Continuous Projection Space (CPS). CPS, which can be seen as a general network embedding technique, vastly outperforms all the co-location, network-based approaches, while retaining a similar interpretability in terms of pairwise distances.

Suggested Citation

  • Andrea Tacchella & Andrea Zaccaria & Marco Miccheli & Luciano Pietronero, 2021. "Relatedness in the Era of Machine Learning," Papers 2103.06017, arXiv.org.
  • Handle: RePEc:arx:papers:2103.06017
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2103.06017
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Frank Neffke & Martin Henning & Ron Boschma, 2011. "How Do Regions Diversify over Time? Industry Relatedness and the Development of New Growth Paths in Regions," Economic Geography, Taylor & Francis Journals, vol. 87(3), pages 237-265, July.
    2. Andrea Zaccaria & Matthieu Cristelli & Andrea Tacchella & Luciano Pietronero, 2014. "How the Taxonomy of Products Drives the Economic Development of Countries," PLOS ONE, Public Library of Science, vol. 9(12), pages 1-17, December.
    3. Cesar A. Hidalgo & Ricardo Hausmann, 2009. "The Building Blocks of Economic Complexity," Papers 0909.3890, arXiv.org.
    4. Manuel Sebastian Mariani & Zhuo-Ming Ren & Jordi Bascompte & Claudio Juan Tessone, 2019. "Nestedness in complex networks: Observation, emergence, and implications," Papers 1905.07593, arXiv.org.
    5. Angelica Sbardella & Emanuele Pugliese & Andrea Zaccaria & Pasquale Scaramozzino, 2018. "The role of complex analysis in modeling economic growth," Papers 1808.10428, arXiv.org.
    6. Giulio Bottazzi & Davide Pirino, 2010. "Measuring Industry Relatedness and Corporate Coherence," LEM Papers Series 2010/10, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    7. Matthieu Cristelli & Andrea Gabrielli & Andrea Tacchella & Guido Caldarelli & Luciano Pietronero, 2013. "Measuring the Intangibles: A Metrics for the Economic Complexity of Countries and Products," PLOS ONE, Public Library of Science, vol. 8(8), pages 1-20, August.
    8. Sebastián Bustos & Charles Gomez & Ricardo Hausmann & César A Hidalgo, 2012. "The Dynamics of Nestedness Predicts the Evolution of Industrial Ecosystems," PLOS ONE, Public Library of Science, vol. 7(11), pages 1-8, November.
    9. Viktor Stojkoski & Zoran Utkovski & Ljupco Kocarev, 2016. "The Impact of Services on Economic Complexity: Service Sophistication as Route for Economic Growth," PLOS ONE, Public Library of Science, vol. 11(8), pages 1-29, August.
    10. Joël Bun & Jean-Philippe Bouchaud & Marc Potters, 2017. "Cleaning large correlation matrices: tools from random matrix theory," Post-Print hal-01491304, HAL.
    11. Alje van Dam & Koen Frenken, 2020. "Vertical vs. Horizontal Policy in a Capabilities Model of Economic Development," Papers in Evolutionary Economic Geography (PEEG) 2037, Utrecht University, Department of Human Geography and Spatial Planning, Group Economic Geography, revised Aug 2020.
    12. Lionel Nesta & Pier Paolo Saviotti, 2005. "Coherence of the Knowledge Base and the Firms’ Innovative Performance. Evidence from the Bio-Pharmaceutical Industry," SciencePo Working papers Main hal-03417696, HAL.
    13. Ms. Natasha X Che, 2020. "Intelligent Export Diversification: An Export Recommendation System with Machine Learning," IMF Working Papers 2020/175, International Monetary Fund.
    14. Fabio Saracco & Mika J. Straka & Riccardo Di Clemente & Andrea Gabrielli & Guido Caldarelli & Tiziano Squartini, 2016. "Inferring monopartite projections of bipartite networks: an entropy-based approach," Papers 1607.02481, arXiv.org, revised May 2017.
    15. C. A. Hidalgo & B. Klinger & A. -L. Barabasi & R. Hausmann, 2007. "The Product Space Conditions the Development of Nations," Papers 0708.2090, arXiv.org.
    16. Pierre-Alexandre Balland & Ron Boschma & Joan Crespo & David L. Rigby, 2019. "Smart specialization policy in the European Union: relatedness, knowledge complexity and regional diversification," Regional Studies, Taylor & Francis Journals, vol. 53(9), pages 1252-1268, September.
    17. Lionel Nesta & Pier Paolo Saviotti, 2005. "Coherence Of The Knowledge Base And The Firm'S Innovative Performance: Evidence From The U.S. Pharmaceutical Industry," Journal of Industrial Economics, Wiley Blackwell, vol. 53(1), pages 123-142, March.
    18. Aamena Alshamsi & Flávio L. Pinheiro & Cesar A. Hidalgo, 2018. "Optimal diversification strategies in the networks of related products and of related research areas," Nature Communications, Nature, vol. 9(1), pages 1-7, December.
    19. David J. Teece & Richard Rumelt & Giovanni Dosi & Sidney Winter, 2000. "Understanding Corporate Coherence: Theory and Evidence," Chapters, in: Innovation, Organization and Economic Dynamics, chapter 9, pages 264-293, Edward Elgar Publishing.
    20. Andrea Palmucci & Hao Liao & Andrea Napoletano & Andrea Zaccaria, 2020. "Where is your field going? A machine learning approach to study the relative motion of the domains of physics," PLOS ONE, Public Library of Science, vol. 15(6), pages 1-16, June.
    21. Tacchella, A. & Cristelli, M. & Caldarelli, G. & Gabrielli, A. & Pietronero, L., 2013. "Economic complexity: Conceptual grounding of a new metrics for global competitiveness," Journal of Economic Dynamics and Control, Elsevier, vol. 37(8), pages 1683-1691.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Dario Mazzilli & Manuel Sebastian Mariani & Flaviano Morone & Aurelio Patelli, 2022. "Equivalence between the Fitness-Complexity and the Sinkhorn-Knopp algorithms," Papers 2212.12356, arXiv.org, revised Mar 2024.
    2. Yang Li & Frank Neffke, 2022. "Relatedness in regional development: in search of the right specification," Papers in Evolutionary Economic Geography (PEEG) 2208, Utrecht University, Department of Human Geography and Spatial Planning, Group Economic Geography, revised Apr 2022.
    3. Bernardo Caldarola & Dario Mazzilli & Lorenzo Napolitano & Aurelio Patelli & Angelica Sbardella, 2023. "Economic complexity and the sustainability transition: A review of data, methods, and literature," Papers 2308.07172, arXiv.org, revised Mar 2024.
    4. Seung Hwan Kim & Bogang Jun & Jeong-Dong Lee, 2023. "Technological relatedness: how do firms diversify their technology?," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(9), pages 4901-4931, September.
    5. Massimiliano Fessina & Giambattista Albora & Andrea Tacchella & Andrea Zaccaria, 2022. "Which products activate a product? An explainable machine learning approach," Papers 2212.03094, arXiv.org.
    6. Sabrina Aufiero & Giordano De Marzo & Angelica Sbardella & Andrea Zaccaria, 2023. "Mapping job complexity and skills into wages," Papers 2304.05251, arXiv.org.
    7. Castañeda, Gonzalo & Pietronero, Luciano & Romero-Padilla, Juan & Zaccaria, Andrea, 2022. "The complex dynamic of growth: Fitness and the different patterns of economic activity in the medium and long terms," Structural Change and Economic Dynamics, Elsevier, vol. 62(C), pages 231-246.
    8. Angelica Sbardella & Andrea Zaccaria & Luciano Pietronero & Pasquale Scaramozzino, 2021. "Behind the Italian Regional Divide: An Economic Fitness and Complexity Perspective," LEM Papers Series 2021/30, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Tacchella, Andrea & Zaccaria, Andrea & Miccheli, Marco & Pietronero, Luciano, 2023. "Relatedness in the era of machine learning," Chaos, Solitons & Fractals, Elsevier, vol. 176(C).
    2. Angelica Sbardella & Andrea Zaccaria & Luciano Pietronero & Pasquale Scaramozzino, 2021. "Behind the Italian Regional Divide: An Economic Fitness and Complexity Perspective," LEM Papers Series 2021/30, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    3. Francesco de Cunzo & Alberto Petri & Andrea Zaccaria & Angelica Sbardella, 2022. "The trickle down from environmental innovation to productive complexity," Papers 2206.07537, arXiv.org.
    4. Balland, Pierre-Alexandre & Broekel, Tom & Diodato, Dario & Giuliani, Elisa & Hausmann, Ricardo & O'Clery, Neave & Rigby, David, 2022. "Reprint of The new paradigm of economic complexity," Research Policy, Elsevier, vol. 51(8).
    5. Hidalgo, César A., 2023. "The policy implications of economic complexity," Research Policy, Elsevier, vol. 52(9).
    6. Sabrina Aufiero & Giordano De Marzo & Angelica Sbardella & Andrea Zaccaria, 2023. "Mapping job complexity and skills into wages," Papers 2304.05251, arXiv.org.
    7. Frank Neffke & Angelica Sbardella & Ulrich Schetter & Andrea Tacchella, 2024. "Economic Complexity Analysis," Papers in Evolutionary Economic Geography (PEEG) 2430, Utrecht University, Department of Human Geography and Spatial Planning, Group Economic Geography, revised Oct 2024.
    8. Balland, Pierre-Alexandre & Broekel, Tom & Diodato, Dario & Giuliani, Elisa & Hausmann, Ricardo & O'Clery, Neave & Rigby, David, 2022. "The new paradigm of economic complexity," Research Policy, Elsevier, vol. 51(3).
    9. Saurabh Mishra & Robert Koopman & Giuditta De-Prato & Anand Rao & Israel Osorio-Rodarte & Julie Kim & Nikola Spatafora & Keith Strier & Andrea Zaccaria, 2021. "AI Specialization for Pathways of Economic Diversification," Papers 2103.11042, arXiv.org.
    10. Hausmann, Ricardo & Stock, Daniel P. & Yıldırım, Muhammed A., 2022. "Implied comparative advantage," Research Policy, Elsevier, vol. 51(8).
    11. Bernardo Caldarola & Dario Mazzilli & Lorenzo Napolitano & Aurelio Patelli & Angelica Sbardella, 2023. "Economic complexity and the sustainability transition: A review of data, methods, and literature," Papers 2308.07172, arXiv.org, revised Mar 2024.
    12. C'esar A. Hidalgo, 2022. "The Policy Implications of Economic Complexity," Papers 2205.02164, arXiv.org, revised Aug 2023.
    13. Massimiliano Fessina & Giambattista Albora & Andrea Tacchella & Andrea Zaccaria, 2022. "Which products activate a product? An explainable machine learning approach," Papers 2212.03094, arXiv.org.
    14. Aurelio Patelli & Andrea Zaccaria & Luciano Pietronero, 2021. "Universal Database for Economic Complexity," Papers 2110.00302, arXiv.org.
    15. Freire, Clovis, 2019. "Economic diversification: A model of structural economic dynamics and endogenous technological change," Structural Change and Economic Dynamics, Elsevier, vol. 49(C), pages 13-28.
    16. Castañeda, Gonzalo & Pietronero, Luciano & Romero-Padilla, Juan & Zaccaria, Andrea, 2022. "The complex dynamic of growth: Fitness and the different patterns of economic activity in the medium and long terms," Structural Change and Economic Dynamics, Elsevier, vol. 62(C), pages 231-246.
    17. Seung Hwan Kim & Bogang Jun & Jeong-Dong Lee, 2023. "Technological relatedness: how do firms diversify their technology?," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(9), pages 4901-4931, September.
    18. Emanuele Pugliese & Lorenzo Napolitano & Andrea Zaccaria & Luciano Pietronero, 2019. "Coherent diversification in corporate technological portfolios," PLOS ONE, Public Library of Science, vol. 14(10), pages 1-22, October.
    19. PUGLIESE Emanuele & TACCHELLA Andrea, 2020. "Economic complexity for competitiveness and innovation: A novel bottom-up strategy linking global and regional capacities," JRC Research Reports JRC122086, Joint Research Centre.
    20. Mealy, Penny & Teytelboym, Alexander, 2017. "Economic Complexity and the Green Economy," INET Oxford Working Papers 2018-03, Institute for New Economic Thinking at the Oxford Martin School, University of Oxford, revised Feb 2019.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2103.06017. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.