IDEAS home Printed from https://ideas.repec.org/a/nat/natcom/v15y2024i1d10.1038_s41467-024-49173-5.html
   My bibliography  Save this article

Shared functional specialization in transformer-based language models and the human brain

Author

Listed:
  • Sreejan Kumar

    (Princeton University)

  • Theodore R. Sumers

    (Princeton University)

  • Takateru Yamakoshi

    (The University of Tokyo)

  • Ariel Goldstein

    (Hebrew University)

  • Uri Hasson

    (Princeton University
    Princeton University)

  • Kenneth A. Norman

    (Princeton University
    Princeton University)

  • Thomas L. Griffiths

    (Princeton University
    Princeton University)

  • Robert D. Hawkins

    (Princeton University
    Princeton University)

  • Samuel A. Nastase

    (Princeton University)

Abstract

When processing language, the brain is thought to deploy specialized computations to construct meaning from complex linguistic structures. Recently, artificial neural networks based on the Transformer architecture have revolutionized the field of natural language processing. Transformers integrate contextual information across words via structured circuit computations. Prior work has focused on the internal representations (“embeddings”) generated by these circuits. In this paper, we instead analyze the circuit computations directly: we deconstruct these computations into the functionally-specialized “transformations” that integrate contextual information across words. Using functional MRI data acquired while participants listened to naturalistic stories, we first verify that the transformations account for considerable variance in brain activity across the cortical language network. We then demonstrate that the emergent computations performed by individual, functionally-specialized “attention heads” differentially predict brain activity in specific cortical regions. These heads fall along gradients corresponding to different layers and context lengths in a low-dimensional cortical space.

Suggested Citation

  • Sreejan Kumar & Theodore R. Sumers & Takateru Yamakoshi & Ariel Goldstein & Uri Hasson & Kenneth A. Norman & Thomas L. Griffiths & Robert D. Hawkins & Samuel A. Nastase, 2024. "Shared functional specialization in transformer-based language models and the human brain," Nature Communications, Nature, vol. 15(1), pages 1-19, December.
  • Handle: RePEc:nat:natcom:v:15:y:2024:i:1:d:10.1038_s41467-024-49173-5
    DOI: 10.1038/s41467-024-49173-5
    as

    Download full text from publisher

    File URL: https://www.nature.com/articles/s41467-024-49173-5
    File Function: Abstract
    Download Restriction: no

    File URL: https://libkey.io/10.1038/s41467-024-49173-5?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Alexander G. Huth & Wendy A. de Heer & Thomas L. Griffiths & Frédéric E. Theunissen & Jack L. Gallant, 2016. "Natural speech reveals the semantic maps that tile human cerebral cortex," Nature, Nature, vol. 532(7600), pages 453-458, April.
    2. Hamed Nili & Cai Wingfield & Alexander Walther & Li Su & William Marslen-Wilson & Nikolaus Kriegeskorte, 2014. "A Toolbox for Representational Similarity Analysis," PLOS Computational Biology, Public Library of Science, vol. 10(4), pages 1-11, April.
    3. Charlotte Caucheteux & Alexandre Gramfort & Jean-Rémi King, 2023. "Evidence of a predictive coding hierarchy in the human brain listening to speech," Nature Human Behaviour, Nature, vol. 7(3), pages 430-441, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Sebastian P. H. Speer & Laetitia Mwilambwe-Tshilobo & Lily Tsoi & Shannon M. Burns & Emily B. Falk & Diana I. Tamir, 2024. "Hyperscanning shows friends explore and strangers converge in conversation," Nature Communications, Nature, vol. 15(1), pages 1-13, December.
    2. Jörn Diedrichsen & Nikolaus Kriegeskorte, 2017. "Representational models: A common framework for understanding encoding, pattern-component, and representational-similarity analysis," PLOS Computational Biology, Public Library of Science, vol. 13(4), pages 1-33, April.
    3. Keiko Ohmae & Shogo Ohmae, 2024. "Emergence of syntax and word prediction in an artificial neural circuit of the cerebellum," Nature Communications, Nature, vol. 15(1), pages 1-13, December.
    4. Valentina Krenz & Arjen Alink & Tobias Sommer & Benno Roozendaal & Lars Schwabe, 2023. "Time-dependent memory transformation in hippocampus and neocortex is semantic in nature," Nature Communications, Nature, vol. 14(1), pages 1-17, December.
    5. Chandan Singh & Armin Askari & Rich Caruana & Jianfeng Gao, 2023. "Augmenting interpretable models with large language models during training," Nature Communications, Nature, vol. 14(1), pages 1-11, December.
    6. Hamed Nili & Alexander Walther & Arjen Alink & Nikolaus Kriegeskorte, 2020. "Inferring exemplar discriminability in brain representations," PLOS ONE, Public Library of Science, vol. 15(6), pages 1-28, June.
    7. Beau Sievers & Christopher Welker & Uri Hasson & Adam M. Kleinbaum & Thalia Wheatley, 2024. "Consensus-building conversation leads to neural alignment," Nature Communications, Nature, vol. 15(1), pages 1-15, December.
    8. Lorenza Lucchi Basili & Pier Luigi Sacco, 2017. "Tie-Up Cycles in Long-Term Mating. Part II: Fictional Narratives and the Social Cognition of Mating," Challenges, MDPI, vol. 8(1), pages 1-60, February.
    9. Ming Bo Cai & Nicolas W Schuck & Jonathan W Pillow & Yael Niv, 2019. "Representational structure or task structure? Bias in neural representational similarity analysis and a Bayesian method for reducing bias," PLOS Computational Biology, Public Library of Science, vol. 15(5), pages 1-30, May.
    10. Desjardins, Christoph, 2021. "Don't be too SMART, but SAVE your goals: Proposal for a renewed goal-setting formula for Generation Y," Journal of Applied Leadership and Management, Hochschule Kempten - University of Applied Sciences, Professional School of Business & Technology, vol. 9, pages 73-87.
    11. Cai Wingfield & Li Su & Xunying Liu & Chao Zhang & Phil Woodland & Andrew Thwaites & Elisabeth Fonteneau & William D Marslen-Wilson, 2017. "Relating dynamic brain states to dynamic machine states: Human and machine solutions to the speech recognition problem," PLOS Computational Biology, Public Library of Science, vol. 13(9), pages 1-25, September.
    12. Maryam Honari-Jahromi & Brea Chouinard & Esti Blanco-Elorrieta & Liina Pylkkänen & Alona Fyshe, 2021. "Neural representation of words within phrases: Temporal evolution of color-adjectives and object-nouns during simple composition," PLOS ONE, Public Library of Science, vol. 16(3), pages 1-17, March.
    13. Xue L. Gong & Alexander G. Huth & Fatma Deniz & Keith Johnson & Jack L. Gallant & Frédéric E. Theunissen, 2023. "Phonemic segmentation of narrative speech in human cerebral cortex," Nature Communications, Nature, vol. 14(1), pages 1-17, December.
    14. Michael F Bonner & Russell A Epstein, 2018. "Computational mechanisms underlying cortical responses to the affordance properties of visual scenes," PLOS Computational Biology, Public Library of Science, vol. 14(4), pages 1-31, April.
    15. Laurent Caplette & Nicholas B. Turk-Browne, 2024. "Computational reconstruction of mental representations using human behavior," Nature Communications, Nature, vol. 15(1), pages 1-19, December.
    16. David M Alexander & Tonio Ball & Andreas Schulze-Bonhage & Cees van Leeuwen, 2019. "Large-scale cortical travelling waves predict localized future cortical signals," PLOS Computational Biology, Public Library of Science, vol. 15(11), pages 1-34, November.
    17. Ariel Goldstein & Avigail Grinstein-Dabush & Mariano Schain & Haocheng Wang & Zhuoqiao Hong & Bobbi Aubrey & Samuel A. Nastase & Zaid Zada & Eric Ham & Amir Feder & Harshvardhan Gazula & Eliav Buchnik, 2024. "Alignment of brain embeddings and artificial contextual embeddings in natural language points to common geometric patterns," Nature Communications, Nature, vol. 15(1), pages 1-12, December.
    18. Máté Aller & Agoston Mihalik & Uta Noppeney, 2022. "Audiovisual adaptation is expressed in spatial and decisional codes," Nature Communications, Nature, vol. 13(1), pages 1-17, December.
    19. Christianne Jacobs & Kirsten Petras & Pieter Moors & Valerie Goffaux, 2020. "Contrast versus identity encoding in the face image follow distinct orientation selectivity profiles," PLOS ONE, Public Library of Science, vol. 15(3), pages 1-22, March.
    20. Satoko Amemori & Ann M. Graybiel & Ken-ichi Amemori, 2024. "Cingulate microstimulation induces negative decision-making via reduced top-down influence on primate fronto-cingulo-striatal network," Nature Communications, Nature, vol. 15(1), pages 1-17, December.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nat:natcom:v:15:y:2024:i:1:d:10.1038_s41467-024-49173-5. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.nature.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.