IDEAS home Printed from https://ideas.repec.org/a/plo/pcbi00/1008228.html
   My bibliography  Save this article

Finding, visualizing, and quantifying latent structure across diverse animal vocal repertoires

Author

Listed:
  • Tim Sainburg
  • Marvin Thielk
  • Timothy Q Gentner

Abstract

Animals produce vocalizations that range in complexity from a single repeated call to hundreds of unique vocal elements patterned in sequences unfolding over hours. Characterizing complex vocalizations can require considerable effort and a deep intuition about each species’ vocal behavior. Even with a great deal of experience, human characterizations of animal communication can be affected by human perceptual biases. We present a set of computational methods for projecting animal vocalizations into low dimensional latent representational spaces that are directly learned from the spectrograms of vocal signals. We apply these methods to diverse datasets from over 20 species, including humans, bats, songbirds, mice, cetaceans, and nonhuman primates. Latent projections uncover complex features of data in visually intuitive and quantifiable ways, enabling high-powered comparative analyses of vocal acoustics. We introduce methods for analyzing vocalizations as both discrete sequences and as continuous latent variables. Each method can be used to disentangle complex spectro-temporal structure and observe long-timescale organization in communication.Author summary: Of the thousands of species that communicate vocally, the repertoires of only a tiny minority have been characterized or studied in detail. This is due, in large part, to traditional analysis methods that require a high level of expertise that is hard to develop and often species-specific. Here, we present a set of unsupervised methods to project animal vocalizations into latent feature spaces to quantitatively compare and develop visual intuitions about animal vocalizations. We demonstrate these methods across a series of analyses over 19 datasets of animal vocalizations from 29 different species, including songbirds, mice, monkeys, humans, and whales. We show how learned latent feature spaces untangle complex spectro-temporal structure, enable cross-species comparisons, and uncover high-level attributes of vocalizations such as stereotypy in vocal element clusters, population regiolects, coarticulation, and individual identity.

Suggested Citation

  • Tim Sainburg & Marvin Thielk & Timothy Q Gentner, 2020. "Finding, visualizing, and quantifying latent structure across diverse animal vocal repertoires," PLOS Computational Biology, Public Library of Science, vol. 16(10), pages 1-48, October.
  • Handle: RePEc:plo:pcbi00:1008228
    DOI: 10.1371/journal.pcbi.1008228
    as

    Download full text from publisher

    File URL: https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1008228
    Download Restriction: no

    File URL: https://journals.plos.org/ploscompbiol/article/file?id=10.1371/journal.pcbi.1008228&type=printable
    Download Restriction: no

    File URL: https://libkey.io/10.1371/journal.pcbi.1008228?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Tim Sainburg & Brad Theilman & Marvin Thielk & Timothy Q. Gentner, 2019. "Parallels in the sequential organization of birdsong and human speech," Nature Communications, Nature, vol. 10(1), pages 1-11, December.
    2. Richard W Hedley, 2016. "Complexity, Predictability and Time Homogeneity of Syntax in the Songs of Cassin’s Vireo (Vireo cassinii)," PLOS ONE, Public Library of Science, vol. 11(4), pages 1-23, April.
    3. Takuya Koumura & Kazuo Okanoya, 2016. "Automatic Recognition of Element Classes and Boundaries in the Birdsong with Variable Sequences," PLOS ONE, Public Library of Science, vol. 11(7), pages 1-24, July.
    4. Kentaro Katahira & Kenta Suzuki & Kazuo Okanoya & Masato Okada, 2011. "Complex Sequencing Rules of Birdsong Can be Explained by Simple Hidden Markov Processes," PLOS ONE, Public Library of Science, vol. 6(9), pages 1-9, September.
    5. Robert F. Lachlan & Oliver Ratmann & Stephen Nowicki, 2018. "Cultural conformity generates extremely stable traditions in bird song," Nature Communications, Nature, vol. 9(1), pages 1-9, December.
    6. Gregg A Castellucci & Daniel Calbick & David McCormick, 2018. "The temporal organization of mouse ultrasonic vocalizations," PLOS ONE, Public Library of Science, vol. 13(10), pages 1-40, October.
    7. Paolo Giudici & Tobias Ryden & Pierre Vandekerkhove, 2000. "Likelihood-Ratio Tests for Hidden Markov Models," Biometrics, The International Biometric Society, vol. 56(3), pages 742-747, September.
    8. Julie E. Elie & Frédéric E. Theunissen, 2018. "Zebra finches identify individuals using vocal signatures unique to each call type," Nature Communications, Nature, vol. 9(1), pages 1-11, December.
    9. Solveig C Mouterde & Frédéric E Theunissen & Julie E Elie & Clémentine Vignal & Nicolas Mathevon, 2014. "Acoustic Communication and Sound Degradation: How Do the Individual Signatures of Male and Female Zebra Finch Calls Transmit over Distance?," PLOS ONE, Public Library of Science, vol. 9(7), pages 1-17, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Margot C Bjoring & C Daniel Meliza, 2019. "A low-threshold potassium current enhances sparseness and reliability in a model of avian auditory cortex," PLOS Computational Biology, Public Library of Science, vol. 15(1), pages 1-20, January.
    2. Heather Williams & Andrew Scharf & Anna R. Ryba & D. Ryan Norris & Daniel J. Mennill & Amy E. M. Newman & Stéphanie M. Doucet & Julie C. Blackwood, 2022. "Cumulative cultural evolution and mechanisms for cultural selection in wild bird songs," Nature Communications, Nature, vol. 13(1), pages 1-11, December.
    3. Jörn Dannemann & Hajo Holzmann, 2008. "Likelihood Ratio Testing for Hidden Markov Models Under Non‐standard Conditions," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 35(2), pages 309-321, June.
    4. Joshua M Mueller & Primoz Ravbar & Julie H Simpson & Jean M Carlson, 2019. "Drosophila melanogaster grooming possesses syntax with distinct rules at different temporal scales," PLOS Computational Biology, Public Library of Science, vol. 15(6), pages 1-25, June.
    5. Roberto Colombi & Sabrina Giordano, 2011. "Testing lumpability for marginal discrete hidden Markov models," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 95(3), pages 293-311, September.
    6. Julie E Elie & Frédéric E Theunissen, 2019. "Invariant neural responses for sensory categories revealed by the time-varying information for communication calls," PLOS Computational Biology, Public Library of Science, vol. 15(9), pages 1-43, September.
    7. Takuto Kawaji & Mizuki Fujibayashi & Kentaro Abe, 2024. "Goal-directed and flexible modulation of syllable sequence within birdsong," Nature Communications, Nature, vol. 15(1), pages 1-12, December.
    8. Peter M C Harrison & Roberta Bianco & Maria Chait & Marcus T Pearce, 2020. "PPM-Decay: A computational model of auditory prediction with memory decay," PLOS Computational Biology, Public Library of Science, vol. 16(11), pages 1-41, November.
    9. A Ivanenko & P Watkins & M A J van Gerven & K Hammerschmidt & B Englitz, 2020. "Classifying sex and strain from mouse ultrasonic vocalizations using deep learning," PLOS Computational Biology, Public Library of Science, vol. 16(6), pages 1-27, June.
    10. Max Greenfeld & Dmitri S Pavlichin & Hideo Mabuchi & Daniel Herschlag, 2012. "Single Molecule Analysis Research Tool (SMART): An Integrated Approach for Analyzing Single Molecule Data," PLOS ONE, Public Library of Science, vol. 7(2), pages 1-12, February.
    11. Mason Youngblood & Joseph M. Stubbersfield & Olivier Morin & Ryan Glassman & Alberto Acerbi, 2023. "Negativity bias in the spread of voter fraud conspiracy theory tweets during the 2020 US election," Palgrave Communications, Palgrave Macmillan, vol. 10(1), pages 1-11, December.
    12. Denton, Kaleda K. & Liberman, Uri & Feldman, Marcus W., 2024. "On random conformity bias in cultural transmission of polychotomous traits," Theoretical Population Biology, Elsevier, vol. 156(C), pages 5-11.
    13. Hugo Loning & Simon C Griffith & Marc Naguib, 2022. "Zebra finch song is a very short-range signal in the wild: evidence from an integrated approach [Song and aggressive signaling in Bachman’s Sparrow]," Behavioral Ecology, International Society for Behavioral Ecology, vol. 33(1), pages 37-46.
    14. Bolano, Danilo & Berchtold, André, 2016. "General framework and model building in the class of Hidden Mixture Transition Distribution models," Computational Statistics & Data Analysis, Elsevier, vol. 93(C), pages 131-145.
    15. Dannemann, Jorn & Holzmann, Hajo, 2008. "The likelihood ratio test for hidden Markov models in two-sample problems," Computational Statistics & Data Analysis, Elsevier, vol. 52(4), pages 1850-1859, January.
    16. Rachel MacKay Altman, 2004. "Assessing the Goodness-of-Fit of Hidden Markov Models," Biometrics, The International Biometric Society, vol. 60(2), pages 444-450, June.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pcbi00:1008228. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ploscompbiol (email available below). General contact details of provider: https://journals.plos.org/ploscompbiol/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.