IDEAS home Printed from https://ideas.repec.org/a/gam/jsusta/v14y2022i21p14324-d961066.html
   My bibliography  Save this article

A Scalable Open-Source Framework for Machine Learning-Based Image Collection, Annotation and Classification: A Case Study for Automatic Fish Species Identification

Author

Listed:
  • Catarina N. S. Silva

    (Nature Research Centre, LT-08412 Vilnius, Lithuania)

  • Justas Dainys

    (Nature Research Centre, LT-08412 Vilnius, Lithuania)

  • Sean Simmons

    (MyCatch and Angler’s Atlas, Prince George, BC V2L 4S1, Canada)

  • Vincentas Vienožinskis

    (Deeper, LT-10312 Vilnius, Lithuania)

  • Asta Audzijonyte

    (Nature Research Centre, LT-08412 Vilnius, Lithuania
    Institute for Marine and Antarctic Studies, University of Tasmania, Hobart 7004, Australia)

Abstract

Citizen science platforms, social media and smart phone applications enable the collection of large amounts of georeferenced images. This provides a huge opportunity in biodiversity and ecological research, but also creates challenges for efficient data handling and processing. Recreational and small-scale fisheries is one of the fields that could be revolutionised by efficient, widely accessible and machine learning-based processing of georeferenced images. Most non-commercial inland and coastal fisheries are considered data poor and are rarely assessed, yet they provide multiple societal benefits and can have substantial ecological impacts. Given that large quantities of georeferenced fish images are being collected by fishers every day, artificial intelligence (AI) and computer vision applications offer a great opportunity to automate their analyses by providing species identification, and potentially also fish size estimation. This would deliver data needed for fisheries management and fisher engagement. To date, however, many AI image analysis applications in fisheries are focused on the commercial sector, limited to specific species or settings, and are not publicly available. In addition, using AI and computer vision tools often requires a strong background in programming. In this study, we aim to facilitate broader use of computer vision tools in fisheries and ecological research by compiling an open-source user friendly and modular framework for large-scale image storage, handling, annotation and automatic classification, using cost- and labour-efficient methodologies. The tool is based on TensorFlow Lite Model Maker library, and includes data augmentation and transfer learning techniques applied to different convolutional neural network models. We demonstrate the potential application of this framework using a small example dataset of fish images taken through a recreational fishing smartphone application. The framework presented here can be used to develop region-specific species identification models, which could potentially be combined into a larger hierarchical model.

Suggested Citation

  • Catarina N. S. Silva & Justas Dainys & Sean Simmons & Vincentas Vienožinskis & Asta Audzijonyte, 2022. "A Scalable Open-Source Framework for Machine Learning-Based Image Collection, Annotation and Classification: A Case Study for Automatic Fish Species Identification," Sustainability, MDPI, vol. 14(21), pages 1-13, November.
    • Handle: RePEc:gam:jsusta:v:14:y:2022:i:21:p:14324-:d:961066
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2071-1050/14/21/14324/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2071-1050/14/21/14324/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Rusul Abduljabbar & Hussein Dia & Sohani Liyanage & Saeed Asadi Bagloee, 2019. "Applications of Artificial Intelligence in Transport: An Overview," Sustainability, MDPI, vol. 11(1), pages 1-24, January.
    2. Sanaz Honarmand Ebrahimi & Marinus Ossewaarde & Ariana Need, 2021. "Smart Fishery: A Systematic Review and Research Agenda for Sustainable Fisheries in the Age of AI," Sustainability, MDPI, vol. 13(11), pages 1-20, May.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Tan Yigitcanlar & Kevin C. Desouza & Luke Butler & Farnoosh Roozkhosh, 2020. "Contributions and Risks of Artificial Intelligence (AI) in Building Smarter Cities: Insights from a Systematic Review of the Literature," Energies, MDPI, vol. 13(6), pages 1-38, March.
    2. Ghasri, Milad & Ardeshiri, Ali & Zhang, Xiang & Waller, S. Travis, 2024. "Analysing preferences for integrated micromobility and public transport systems: A hierarchical latent class approach considering taste heterogeneity and attribute non-attendance," Transportation Research Part A: Policy and Practice, Elsevier, vol. 181(C).
    3. Wu, Min & Wang, Nanxi & Yuen, Kum Fai, 2023. "Can autonomy level and anthropomorphic characteristics affect public acceptance and trust towards shared autonomous vehicles?," Technological Forecasting and Social Change, Elsevier, vol. 189(C).
    4. Mochen Liao & Kai Lan & Yuan Yao, 2022. "Sustainability implications of artificial intelligence in the chemical industry: A conceptual framework," Journal of Industrial Ecology, Yale University, vol. 26(1), pages 164-182, February.
    5. Zulamir Hassani, Afdhal & Yusoff, Fazirah & Wan Zain, Wan Nor Aisyah, 2021. "Fair and Responsible in Logistics IR 4.0," MPRA Paper 108432, University Library of Munich, Germany.
    6. Marya Butt & Ander de Keijzer, 2022. "Using Transfer Learning to Train a Binary Classifier for Lorrca Ektacytometery Microscopic Images of Sickle Cells and Healthy Red Blood Cells," Data, MDPI, vol. 7(9), pages 1-21, September.
    7. Sohani Liyanage & Hussein Dia & Rusul Abduljabbar & Saeed Asadi Bagloee, 2019. "Flexible Mobility On-Demand: An Environmental Scan," Sustainability, MDPI, vol. 11(5), pages 1-39, February.
    8. Pamucar, Dragan & Deveci, Muhammet & Gokasar, Ilgin & Tavana, Madjid & Köppen, Mario, 2022. "A metaverse assessment model for sustainable transportation using ordinal priority approach and Aczel-Alsina norms," Technological Forecasting and Social Change, Elsevier, vol. 182(C).
    9. Jae-joon Chung & Hyun-Jung Kim, 2020. "An Automobile Environment Detection System Based on Deep Neural Network and its Implementation Using IoT-Enabled In-Vehicle Air Quality Sensors," Sustainability, MDPI, vol. 12(6), pages 1-17, March.
    10. Jin, Hui & Liu, Yue & Wu, Telan & Zhang, Yanpei, 2022. "Site-specific optimization of bus stop locations and designs over a corridor," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 599(C).
    11. Tahmineh Ladi & Shaghayegh Jabalameli & Ayyoob Sharifi, 2022. "Applications of machine learning and deep learning methods for climate change mitigation and adaptation," Environment and Planning B, , vol. 49(4), pages 1314-1330, May.
    12. Eugène Loos & Maria Sourbati & Frauke Behrendt, 2020. "The Role of Mobility Digital Ecosystems for Age-Friendly Urban Public Transport: A Narrative Literature Review," IJERPH, MDPI, vol. 17(20), pages 1-16, October.
    13. Fosso Wamba, Samuel & Bawack, Ransome Epie & Guthrie, Cameron & Queiroz, Maciel M. & Carillo, Kevin Daniel André, 2021. "Are we preparing for a good AI society? A bibliometric review and research agenda," Technological Forecasting and Social Change, Elsevier, vol. 164(C).
    14. De Obesso Arias, María de las Mercedes & Pérez Rivero, Carlos Alberto & Carrero Márquez, Oliver, 2023. "Artificial intelligence to manage workplace bullying," Journal of Business Research, Elsevier, vol. 160(C).
    15. Suleiman Hassan Otuoze & Dexter V. L. Hunt & Ian Jefferson, 2021. "Neural Network Approach to Modelling Transport System Resilience for Major Cities: Case Studies of Lagos and Kano (Nigeria)," Sustainability, MDPI, vol. 13(3), pages 1-20, January.
    16. Maryann Osadebamwen Asemota, 2023. "Facial Recognition Technology For Recruitment In The Russian Workplace," HSE Working papers WP BRP 126/STI/2023, National Research University Higher School of Economics.
    17. Victor Anderson Hodibert & Adriana Narkwa Anderson (PhD) & Kate Neequaye, 2024. "AI/Robotics in the Tourism and Hospitality Sector: Technological Realities and Imaginaries in the Ghanaian Context," International Journal of Research and Innovation in Social Science, International Journal of Research and Innovation in Social Science (IJRISS), vol. 8(1), pages 2471-2480, January.
    18. Tomas Ramirez-Guerrero & Mauricio Toro & Marta S. Tabares & Ricardo Salazar-Cabrera & Álvaro Pachón de la Cruz, 2022. "Key Aspects for IT-Services Integration in Urban Transit Service of Medium-Sized Cities: A Qualitative Exploratory Study in Colombia," Sustainability, MDPI, vol. 14(5), pages 1-20, February.
    19. Fahad Alrukaibi & Rushdi Alsaleh & Tarek Sayed, 2019. "Applying Machine Learning and Statistical Approaches for Travel Time Estimation in Partial Network Coverage," Sustainability, MDPI, vol. 11(14), pages 1-18, July.
    20. Sesil Koutra & Christos S. Ioakimidis, 2022. "Unveiling the Potential of Machine Learning Applications in Urban Planning Challenges," Land, MDPI, vol. 12(1), pages 1-19, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jsusta:v:14:y:2022:i:21:p:14324-:d:961066. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.