IDEAS home Printed from https://ideas.repec.org/a/eee/tefoso/v207y2024ics0040162524004165.html
   My bibliography  Save this article

Measuring digitalization at scale using web scraped data

Author

Listed:
  • Ashouri, Sajad
  • Hajikhani, Arash
  • Suominen, Arho
  • Pukelis, Lukas
  • Cunningham, Scott W.

Abstract

Measuring digitalization has been a central topic in academic discourse. While evaluating firms' efforts in increasing digitalization is crucial, quantifying it at scale, presents considerable challenges. This paper uses website information as a source of data to operationalize a measure of digitalization. Drawing on a sample of 60,942 firms, our approach proposes two distinct measures of digitalization: one at the product level and the other at the general organizational level. We substantiate these measures using a blend of qualitative and quantitative methods. The study validates the content of websites as a relevant source of innovation indicator data and verifies the indicators using multiple experiments. The developed digitalization indicators offer future research an empirical measure of digitalization that can be run at scale, across industries and regions through time.

Suggested Citation

  • Ashouri, Sajad & Hajikhani, Arash & Suominen, Arho & Pukelis, Lukas & Cunningham, Scott W., 2024. "Measuring digitalization at scale using web scraped data," Technological Forecasting and Social Change, Elsevier, vol. 207(C).
    • Handle: RePEc:eee:tefoso:v:207:y:2024:i:c:s0040162524004165
    DOI: 10.1016/j.techfore.2024.123618
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0040162524004165
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.techfore.2024.123618?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Kässi, Otto & Lehdonvirta, Vili, 2018. "Online labour index: Measuring the online gig economy for policy and research," Technological Forecasting and Social Change, Elsevier, vol. 137(C), pages 241-248.
    2. Lee G. Branstetter & Matej Drev & Namho Kwon, 2019. "Get with the Program: Software-Driven Innovation in Traditional Manufacturing," Management Science, INFORMS, vol. 65(2), pages 541-558, February.
    3. Rammer, Christian & Es-Sadki, Nordine, 2023. "Using big data for generating firm-level innovation indicators - a literature review," Technological Forecasting and Social Change, Elsevier, vol. 197(C).
    4. Youngjin Yoo & Richard J. Boland & Kalle Lyytinen & Ann Majchrzak, 2012. "Organizing for Innovation in the Digitized World," Organization Science, INFORMS, vol. 23(5), pages 1398-1408, October.
    5. Guandalini, Ilaria, 2022. "Sustainability through digital transformation: A systematic literature review for research guidance," Journal of Business Research, Elsevier, vol. 148(C), pages 456-471.
    6. Annarelli, Alessandro & Battistella, Cinzia & Nonino, Fabio & Parida, Vinit & Pessot, Elena, 2021. "Literature review on digitalization capabilities: Co-citation analysis of antecedents, conceptualization and consequences," Technological Forecasting and Social Change, Elsevier, vol. 166(C).
    7. Sanjay K. Arora & Yin Li & Jan Youtie & Philip Shapira, 2020. "Measuring dynamic capabilities in new ventures: exploring strategic change in US green goods manufacturing using website data," The Journal of Technology Transfer, Springer, vol. 45(5), pages 1451-1480, October.
    8. Kollmann, Tobias & Stöckmann, Christoph & Niemand, Thomas & Hensellek, Simon & de Cruppe, Katharina, 2021. "A configurational approach to entrepreneurial orientation and cooperation explaining product/service innovation in digital vs. non-digital startups," Journal of Business Research, Elsevier, vol. 125(C), pages 508-519.
    9. Sanjay K. Arora & Jan Youtie & Philip Shapira & Lidan Gao & TingTing Ma, 2013. "Entry strategies in an emerging technology: a pilot web-based study of graphene firms," Scientometrics, Springer;Akadémiai Kiadó, vol. 95(3), pages 1189-1207, June.
    10. Nambisan, Satish & Wright, Mike & Feldman, Maryann, 2019. "The digital transformation of innovation and entrepreneurship: Progress, challenges and key themes," Research Policy, Elsevier, vol. 48(8), pages 1-1.
    11. Jan Lepoutre & Aimé Heene, 2006. "Investigating the Impact of Firm Size on Small Business Social Responsibility: A Critical Review," Journal of Business Ethics, Springer, vol. 67(3), pages 257-273, September.
    12. Grant, Delvin & Yeo, Benjamin, 2018. "A global perspective on tech investment, financing, and ICT on manufacturing and service industry performance," International Journal of Information Management, Elsevier, vol. 43(C), pages 130-145.
    13. Razzaq, Asif & Yang, Xiaodong, 2023. "Digital finance and green growth in China: Appraising inclusive digital finance using web crawler technology and big data," Technological Forecasting and Social Change, Elsevier, vol. 188(C).
    14. Nathan, Max & Rosso, Anna, 2022. "Innovative events: product launches, innovation and firm performance," Research Policy, Elsevier, vol. 51(1).
    15. Kahle, Júlia Hofmeister & Marcon, Érico & Ghezzi, Antonio & Frank, Alejandro G., 2020. "Smart Products value creation in SMEs innovation ecosystems," Technological Forecasting and Social Change, Elsevier, vol. 156(C).
    16. Kayser, Victoria & Blind, Knut, 2017. "Extending the knowledge base of foresight: The contribution of text mining," Technological Forecasting and Social Change, Elsevier, vol. 116(C), pages 208-215.
    17. Blichfeldt, Henrik & Faullant, Rita, 2021. "Performance effects of digital technology adoption and product & service innovation – A process-industry perspective," Technovation, Elsevier, vol. 105(C).
    18. Ann Bartel & Casey Ichniowski & Kathryn Shaw, 2007. "How Does Information Technology Affect Productivity? Plant-Level Comparisons of Product Innovation, Process Improvement, and Worker Skills," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 122(4), pages 1721-1758.
    19. Mohamed A.K. Basuony & Ehab K.A. Mohamed & Ahmed Elragal & Khaled Hussainey, 2020. "Big data analytics of corporate internet disclosures," Accounting Research Journal, Emerald Group Publishing Limited, vol. 35(1), pages 4-20, May.
    20. Balsmeier, Benjamin & Woerter, Martin, 2019. "Is this time different? How digitalization influences job creation and destruction," Research Policy, Elsevier, vol. 48(8), pages 1-1.
    21. Dahlke, Johannes & Bogner, Kristina & Becker, Maike & Schlaile, Michael P. & Pyka, Andreas & Ebersberger, Bernd, 2021. "Crisis-driven innovation and fundamental human needs: A typological framework of rapid-response COVID-19 innovations," Technological Forecasting and Social Change, Elsevier, vol. 169(C).
    22. Flavio Calvino & Chiara Criscuolo & Luca Marcolin & Mariagrazia Squicciarini, 2018. "A taxonomy of digital intensive sectors," OECD Science, Technology and Industry Working Papers 2018/14, OECD Publishing.
    23. Timothy F. Bresnahan & Erik Brynjolfsson & Lorin M. Hitt, 2002. "Information Technology, Workplace Organization, and the Demand for Skilled Labor: Firm-Level Evidence," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 117(1), pages 339-376.
    24. Denicolai, Stefano & Zucchella, Antonella & Magnani, Giovanna, 2021. "Internationalization, digitalization, and sustainability: Are SMEs ready? A survey on synergies and substituting effects among growth paths," Technological Forecasting and Social Change, Elsevier, vol. 166(C).
    25. Kohtamäki, Marko & Parida, Vinit & Oghazi, Pejvak & Gebauer, Heiko & Baines, Tim, 2019. "Digital servitization business models in ecosystems: A theory of the firm," Journal of Business Research, Elsevier, vol. 104(C), pages 380-392.
    26. Cheng, Cong & Wang, Limin, 2022. "How companies configure digital innovation attributes for business model innovation? A configurational view," Technovation, Elsevier, vol. 112(C).
    27. Jan Kinne & Janna Axenbeck, 2020. "Web mining for innovation ecosystem mapping: a framework and a large-scale pilot study," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2011-2041, December.
    28. Coralie Gagne & Sophie Veilleux & Fabiano Armellini & Patrick Cohendet & Luc Sirois, 2023. "Developing Indicators Of Open Innovation Event Outcomes," International Journal of Innovation Management (ijim), World Scientific Publishing Co. Pte. Ltd., vol. 27(03n04), pages 1-45, May.
    29. Christine Legner & Torsten Eymann & Thomas Hess & Christian Matt & Tilo Böhmann & Paul Drews & Alexander Mädche & Nils Urbach & Frederik Ahlemann, 2017. "Digitalization: Opportunity and Challenge for the Business and Information Systems Engineering Community," Business & Information Systems Engineering: The International Journal of WIRTSCHAFTSINFORMATIK, Springer;Gesellschaft für Informatik e.V. (GI), vol. 59(4), pages 301-308, August.
    30. Blazquez, Desamparados & Domenech, Josep, 2018. "Big Data sources and methods for social and economic analyses," Technological Forecasting and Social Change, Elsevier, vol. 130(C), pages 99-113.
    31. Abdullah Gök & Alec Waterworth & Philip Shapira, 2015. "Use of web mining in studying innovation," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(1), pages 653-671, January.
    32. Stefan Schweikl & Robert Obermaier, 2020. "Lessons from three decades of IT productivity research: towards a better understanding of IT-induced productivity effects," Management Review Quarterly, Springer, vol. 70(4), pages 461-507, November.
    33. Li, Yin & Arora, Sanjay & Youtie, Jan & Shapira, Philip, 2018. "Using web mining to explore Triple Helix influences on growth in small and mid-size firms," Technovation, Elsevier, vol. 76, pages 3-14.
    34. Mikalef, Patrick & Boura, Maria & Lekakos, George & Krogstie, John, 2019. "Big data analytics and firm performance: Findings from a mixed-method approach," Journal of Business Research, Elsevier, vol. 98(C), pages 261-276.
    35. Sinan Aral & Peter Weill, 2007. "IT Assets, Organizational Capabilities, and Firm Performance: How Resource Allocations and Organizational Differences Explain Performance Variation," Organization Science, INFORMS, vol. 18(5), pages 763-780, October.
    36. Vendrell-Herrero, Ferran & Bustinza, Oscar F. & Opazo-Basaez, Marco & Gomes, Emanuel, 2023. "Treble innovation firms: Antecedents, outcomes, and enhancing factors," International Journal of Production Economics, Elsevier, vol. 255(C).
    37. Jan Kinne & David Lenz, 2021. "Predicting innovative firms using web mining and deep learning," PLOS ONE, Public Library of Science, vol. 16(4), pages 1-18, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Bottai, Carlo & Crosato, Lisa & Domenech, Josep & Guerzoni, Marco & Liberati, Caterina, 2024. "Scraping innovativeness from corporate websites: Empirical evidence on Italian manufacturing SMEs," Technological Forecasting and Social Change, Elsevier, vol. 207(C).
    2. Schubert, Torben & Ashouri, Sajad & Deschryvere, Matthias & Jäger, Angela & Visentin, Fabiana & Cunningham, Scott & Hajikhani, Arash & Pukelis, Lukas & Suominen, Arho, 2023. "The role of product digitization for productivity," MERIT Working Papers 2023-004, United Nations University - Maastricht Economic and Social Research Institute on Innovation and Technology (MERIT).
    3. Rammer, Christian & Es-Sadki, Nordine, 2023. "Using big data for generating firm-level innovation indicators - a literature review," Technological Forecasting and Social Change, Elsevier, vol. 197(C).
    4. Axenbeck, Janna & Breithaupt, Patrick, 2022. "Measuring the digitalisation of firms: A novel text mining approach," ZEW Discussion Papers 22-065, ZEW - Leibniz Centre for European Economic Research.
    5. Lihua Chen & Yilang Chen, 2023. "A Metaorganizations Perspective on Digital Innovation and Corporate Social Responsibility: Evidence from China," Sustainability, MDPI, vol. 15(14), pages 1-19, July.
    6. Christoph Stich & Emmanouil Tranos & Max Nathan, 2023. "Modeling clusters from the ground up: A web data approach," Environment and Planning B, , vol. 50(1), pages 244-267, January.
    7. Deon Montasser & Ruslan Prijadi & Tengku Ezni Balqiah, 2023. "The Mediating Effect of IT-Enabled Dynamic Capabilities and Organizational Readiness on the Relationship Between Transformational Leadership and Digital Business Model Innovation: Evidence From Indone," SAGE Open, , vol. 13(2), pages 21582440231, June.
    8. Dörr, Julian Oliver & Kinne, Jan & Lenz, David & Licht, Georg & Winker, Peter, 2021. "An integrated data framework for policy guidance in times of dynamic economic shocks," ZEW Discussion Papers 21-062, ZEW - Leibniz Centre for European Economic Research.
    9. Tang, Haodan & Fang, Senhui & Jiang, Dianchun, 2022. "The market value effect of digital mergers and acquisitions: Evidence from China," Economic Modelling, Elsevier, vol. 116(C).
    10. Jia, Yibo & Cui, Li & Su, Jingqin & Wu, Lin & Akter, Shahriar & Kumar, Ajay, 2024. "Digital servitization in digital enterprise: Leveraging digital platform capabilities to unlock data value," International Journal of Production Economics, Elsevier, vol. 278(C).
    11. Zand, Fardad & Van Beers, Cees & Van Leeuwen, George, 2011. "Information technology, organizational change and firm productivity: A panel study of complementarity effects and clustering patterns in Manufacturing and Services," MPRA Paper 46469, University Library of Munich, Germany.
    12. Kroh, Julia & Globocnik, Dietfried & Schultz, Carsten & Holdhof, Frederike & Salomo, Søren, 2024. "Micro-foundations of digital innovation capability – A mixed method approach to develop and validate a multi-dimensional measurement instrument," Technological Forecasting and Social Change, Elsevier, vol. 198(C).
    13. Yaru Li & Qifan Zhang, 2024. "Corporate Digital Transformation and the Internationalization of R&D," Sustainability, MDPI, vol. 16(21), pages 1-20, October.
    14. Lai, Xiaobing & Yue, Shujing & Guo, Chong & Gao, Peng, 2024. "Unleashing global potential: The impact of digital technology innovation on corporate international diversification," Technological Forecasting and Social Change, Elsevier, vol. 208(C).
    15. Kim, Sung Min & Mahoney, Joseph T., 2008. "Resource Co-specialization, Firm Growth, and Organizational Performance: An Empirical Analysis of Organizational Restructuring and IT Implementations," Working Papers 08-0107, University of Illinois at Urbana-Champaign, College of Business.
    16. Sinan Aral & Erik Brynjolfsson & Lynn Wu, 2012. "Three-Way Complementarities: Performance Pay, Human Resource Analytics, and Information Technology," Management Science, INFORMS, vol. 58(5), pages 913-931, May.
    17. Viete, Steffen & Erdsiek, Daniel, 2018. "Trust-based work time and the productivity effects of mobile information technologies in the workplace," ZEW Discussion Papers 18-013, ZEW - Leibniz Centre for European Economic Research.
    18. Peng Huang & Marco Ceccagnoli & Chris Forman & D.J. Wu, 2022. "IT Knowledge Spillovers, Absorptive Capacity, and Productivity: Evidence from Enterprise Software," Information Systems Research, INFORMS, vol. 33(3), pages 908-934, September.
    19. Occhini, Giulia & Tranos, Emmanouil & Wolf, Levi John, 2023. "Occupational segregation in the digital economy? A Natural Language Processing approach using UK Web Data," SocArXiv z8xta, Center for Open Science.
    20. Motohashi, Kazuyuki & Zhu, Chen, 2023. "Identifying technology opportunity using dual-attention model and technology-market concordance matrix," Technological Forecasting and Social Change, Elsevier, vol. 197(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:tefoso:v:207:y:2024:i:c:s0040162524004165. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.sciencedirect.com/science/journal/00401625 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.