IDEAS home Printed from https://ideas.repec.org/a/bla/jinfst/v74y2023i11p1307-1324.html
   My bibliography  Save this article

Do you see what I see? Measuring the semantic differences in image‐recognition services' outputs

Author

Listed:
  • Anton Berg
  • Matti Nelimarkka

Abstract

As scholars increasingly undertake large‐scale analysis of visual materials, advanced computational tools show promise for informing that process. One technique in the toolbox is image recognition, made readily accessible via Google Vision AI, Microsoft Azure Computer Vision, and Amazon's Rekognition service. However, concerns about such issues as bias factors and low reliability have led to warnings against research employing it. A systematic study of cross‐service label agreement concretized such issues: using eight datasets, spanning professionally produced and user‐generated images, the work showed that image‐recognition services disagree on the most suitable labels for images. Beyond supporting caveats expressed in prior literature, the report articulates two mitigation strategies, both involving the use of multiple image‐recognition services: Highly explorative research could include all the labels, accepting noisier but less restrictive analysis output. Alternatively, scholars may employ word‐embedding‐based approaches to identify concepts that are similar enough for their purposes, then focus on those labels filtered in.

Suggested Citation

  • Anton Berg & Matti Nelimarkka, 2023. "Do you see what I see? Measuring the semantic differences in image‐recognition services' outputs," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 74(11), pages 1307-1324, November.
    • Handle: RePEc:bla:jinfst:v:74:y:2023:i:11:p:1307-1324
    DOI: 10.1002/asi.24827
    as

    Download full text from publisher

    File URL: https://doi.org/10.1002/asi.24827
    Download Restriction: no
    File URL: https://libkey.io/10.1002/asi.24827?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Denny, Matthew J. & Spirling, Arthur, 2018. "Text Preprocessing For Unsupervised Learning: Why It Matters, When It Misleads, And What To Do About It," Political Analysis, Cambridge University Press, vol. 26(2), pages 168-189, April.
    2. Mike Thelwall & Olga Goriunova & Farida Vis & Simon Faulkner & Anne Burns & Jim Aulich & Amalia Mas-Bleda & Emma Stuart & Francesco D'Orazio, 2016. "Chatting through pictures? A classification of images tweeted in one week in the UK and USA," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 67(11), pages 2575-2586, November.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mohamed M. Mostafa, 2023. "A one-hundred-year structural topic modeling analysis of the knowledge structure of international management research," Quality & Quantity: International Journal of Methodology, Springer, vol. 57(4), pages 3905-3935, August.
    2. Latifi, Albina & Naboka-Krell, Viktoriia & Tillmann, Peter & Winker, Peter, 2024. "Fiscal policy in the Bundestag: Textual analysis and macroeconomic effects," European Economic Review, Elsevier, vol. 168(C).
    3. Purwoko Haryadi Santoso & Edi Istiyono & Haryanto & Wahyu Hidayatulloh, 2022. "Thematic Analysis of Indonesian Physics Education Research Literature Using Machine Learning," Data, MDPI, vol. 7(11), pages 1-41, October.
    4. Camilla Salvatore & Silvia Biffignandi & Annamaria Bianchi, 2022. "Corporate Social Responsibility Activities Through Twitter: From Topic Model Analysis to Indexes Measuring Communication Characteristics," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 164(3), pages 1217-1248, December.
    5. Jason Anastasopoulos & George J. Borjas & Gavin G. Cook & Michael Lachanski, 2018. "Job Vacancies, the Beveridge Curve, and Supply Shocks: The Frequency and Content of Help-Wanted Ads in Pre- and Post-Mariel Miami," NBER Working Papers 24580, National Bureau of Economic Research, Inc.
    6. W. Benedikt Schmal, 2024. "Academic Knowledge: Does it Reflect the Combinatorial Growth of Technology?," Papers 2409.20282, arXiv.org.
    7. Lorien Stice-Lawrence, 2022. "Practical issues to consider when working with big data," Review of Accounting Studies, Springer, vol. 27(3), pages 1117-1124, September.
    8. Tyler Andrew Scott & Nicola Ulibarri & Omar Perez Figueroa, 2020. "NEPA and National Trends in Federal Infrastructure Siting in the United States," Review of Policy Research, Policy Studies Organization, vol. 37(5), pages 605-633, September.
    9. Seraphine F. Maerz & Carsten Q. Schneider, 2020. "Comparing public communication in democracies and autocracies: automated text analyses of speeches by heads of government," Quality & Quantity: International Journal of Methodology, Springer, vol. 54(2), pages 517-545, April.
    10. Kim, Da Yeon & Kim, Sang Yong, 2022. "The impact of customer-generated evaluation information on sales in online platform-based markets," Journal of Retailing and Consumer Services, Elsevier, vol. 68(C).
    11. Iasmin Goes, 2023. "Examining the effect of IMF conditionality on natural resource policy," Economics and Politics, Wiley Blackwell, vol. 35(1), pages 227-285, March.
    12. Shah Jahan Miah & Huy Quan Vu & Damminda Alahakoon, 2022. "A social media analytics perspective for human‐oriented smart city planning and management," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 73(1), pages 119-135, January.
    13. Yeomans, Michael, 2021. "A concrete example of construct construction in natural language," Organizational Behavior and Human Decision Processes, Elsevier, vol. 162(C), pages 81-94.
    14. Ai-Hsuan Chiang & Silvana Trimi & Tun-Chih Kou, 2024. "Critical Factors for Implementing Smart Manufacturing: A Supply Chain Perspective," Sustainability, MDPI, vol. 16(22), pages 1-21, November.
    15. Kaiser, Carolin & Ahuvia, Aaron & Rauschnabel, Philipp A. & Wimble, Matt, 2020. "Social media monitoring: What can marketers learn from Facebook brand photos?," Journal of Business Research, Elsevier, vol. 117(C), pages 707-717.
    16. Karell, Daniel & Freedman, Michael Raphael, 2019. "Rhetorics of Radicalism," SocArXiv yfzsh, Center for Open Science.
    17. Bastiaan Bruinsma & Moa Johansson, 2024. "Finding the structure of parliamentary motions in the Swedish Riksdag 1971–2015," Quality & Quantity: International Journal of Methodology, Springer, vol. 58(4), pages 3275-3301, August.
    18. Josephine Lukito & Jason Greenfield & Yunkang Yang & Ross Dahlke & Megan A. Brown & Rebecca Lewis & Bin Chen, 2024. "Audio-as-Data Tools: Replicating Computational Data Processing," Media and Communication, Cogitatio Press, vol. 12.
    19. Justyna Klejdysz & Robin L. Lumsdaine, 2023. "Shifts in ECB Communication: A Textual Analysis of the Press Conference," International Journal of Central Banking, International Journal of Central Banking, vol. 19(2), pages 473-542, June.
    20. Javier De la Hoz-M & Mª José Fernández-Gómez & Susana Mendes, 2021. "LDAShiny: An R Package for Exploratory Review of Scientific Literature Based on a Bayesian Probabilistic Model and Machine Learning Tools," Mathematics, MDPI, vol. 9(14), pages 1-21, July.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jinfst:v:74:y:2023:i:11:p:1307-1324. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www.asis.org .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.