IDEAS home Printed from https://ideas.repec.org/a/nat/natcom/v15y2024i1d10.1038_s41467-024-49173-5.html
   My bibliography  Save this article

Shared functional specialization in transformer-based language models and the human brain

Author

Listed:
  • Sreejan Kumar

    (Princeton University)

  • Theodore R. Sumers

    (Princeton University)

  • Takateru Yamakoshi

    (The University of Tokyo)

  • Ariel Goldstein

    (Hebrew University)

  • Uri Hasson

    (Princeton University
    Princeton University)

  • Kenneth A. Norman

    (Princeton University
    Princeton University)

  • Thomas L. Griffiths

    (Princeton University
    Princeton University)

  • Robert D. Hawkins

    (Princeton University
    Princeton University)

  • Samuel A. Nastase

    (Princeton University)

Abstract

When processing language, the brain is thought to deploy specialized computations to construct meaning from complex linguistic structures. Recently, artificial neural networks based on the Transformer architecture have revolutionized the field of natural language processing. Transformers integrate contextual information across words via structured circuit computations. Prior work has focused on the internal representations (“embeddings”) generated by these circuits. In this paper, we instead analyze the circuit computations directly: we deconstruct these computations into the functionally-specialized “transformations” that integrate contextual information across words. Using functional MRI data acquired while participants listened to naturalistic stories, we first verify that the transformations account for considerable variance in brain activity across the cortical language network. We then demonstrate that the emergent computations performed by individual, functionally-specialized “attention heads” differentially predict brain activity in specific cortical regions. These heads fall along gradients corresponding to different layers and context lengths in a low-dimensional cortical space.

Suggested Citation

  • Sreejan Kumar & Theodore R. Sumers & Takateru Yamakoshi & Ariel Goldstein & Uri Hasson & Kenneth A. Norman & Thomas L. Griffiths & Robert D. Hawkins & Samuel A. Nastase, 2024. "Shared functional specialization in transformer-based language models and the human brain," Nature Communications, Nature, vol. 15(1), pages 1-19, December.
  • Handle: RePEc:nat:natcom:v:15:y:2024:i:1:d:10.1038_s41467-024-49173-5
    DOI: 10.1038/s41467-024-49173-5
    as

    Download full text from publisher

    File URL: https://www.nature.com/articles/s41467-024-49173-5
    File Function: Abstract
    Download Restriction: no

    File URL: https://libkey.io/10.1038/s41467-024-49173-5?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Alexander G. Huth & Wendy A. de Heer & Thomas L. Griffiths & Frédéric E. Theunissen & Jack L. Gallant, 2016. "Natural speech reveals the semantic maps that tile human cerebral cortex," Nature, Nature, vol. 532(7600), pages 453-458, April.
    2. Charlotte Caucheteux & Alexandre Gramfort & Jean-Rémi King, 2023. "Evidence of a predictive coding hierarchy in the human brain listening to speech," Nature Human Behaviour, Nature, vol. 7(3), pages 430-441, March.
    3. Hamed Nili & Cai Wingfield & Alexander Walther & Li Su & William Marslen-Wilson & Nikolaus Kriegeskorte, 2014. "A Toolbox for Representational Similarity Analysis," PLOS Computational Biology, Public Library of Science, vol. 10(4), pages 1-11, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Keiko Ohmae & Shogo Ohmae, 2024. "Emergence of syntax and word prediction in an artificial neural circuit of the cerebellum," Nature Communications, Nature, vol. 15(1), pages 1-13, December.
    2. Sebastian P. H. Speer & Laetitia Mwilambwe-Tshilobo & Lily Tsoi & Shannon M. Burns & Emily B. Falk & Diana I. Tamir, 2024. "Hyperscanning shows friends explore and strangers converge in conversation," Nature Communications, Nature, vol. 15(1), pages 1-13, December.
    3. Jörn Diedrichsen & Nikolaus Kriegeskorte, 2017. "Representational models: A common framework for understanding encoding, pattern-component, and representational-similarity analysis," PLOS Computational Biology, Public Library of Science, vol. 13(4), pages 1-33, April.
    4. Valentina Krenz & Arjen Alink & Tobias Sommer & Benno Roozendaal & Lars Schwabe, 2023. "Time-dependent memory transformation in hippocampus and neocortex is semantic in nature," Nature Communications, Nature, vol. 14(1), pages 1-17, December.
    5. Satoko Amemori & Ann M. Graybiel & Ken-ichi Amemori, 2024. "Cingulate microstimulation induces negative decision-making via reduced top-down influence on primate fronto-cingulo-striatal network," Nature Communications, Nature, vol. 15(1), pages 1-17, December.
    6. Julia Berezutskaya & Zachary V Freudenburg & Umut Güçlü & Marcel A J van Gerven & Nick F Ramsey, 2020. "Brain-optimized extraction of complex sound features that drive continuous auditory perception," PLOS Computational Biology, Public Library of Science, vol. 16(7), pages 1-34, July.
    7. Chandan Singh & Armin Askari & Rich Caruana & Jianfeng Gao, 2023. "Augmenting interpretable models with large language models during training," Nature Communications, Nature, vol. 14(1), pages 1-11, December.
    8. Manoj Kumar & Cameron T Ellis & Qihong Lu & Hejia Zhang & Mihai Capotă & Theodore L Willke & Peter J Ramadge & Nicholas B Turk-Browne & Kenneth A Norman, 2020. "BrainIAK tutorials: User-friendly learning materials for advanced fMRI analysis," PLOS Computational Biology, Public Library of Science, vol. 16(1), pages 1-12, January.
    9. Katherine Farrow & Gilles Grolleau & Naoufel Mzoughi, 2018. "What in the Word! The Scope for the Effect of Word Choice on Economic Behavior," Kyklos, Wiley Blackwell, vol. 71(4), pages 557-580, November.
    10. Hamed Nili & Alexander Walther & Arjen Alink & Nikolaus Kriegeskorte, 2020. "Inferring exemplar discriminability in brain representations," PLOS ONE, Public Library of Science, vol. 15(6), pages 1-28, June.
    11. Sam V Norman-Haignere & Josh H McDermott, 2018. "Neural responses to natural and model-matched stimuli reveal distinct computations in primary and nonprimary auditory cortex," PLOS Biology, Public Library of Science, vol. 16(12), pages 1-46, December.
    12. Katherine R. Storrs & Barton L. Anderson & Roland W. Fleming, 2021. "Unsupervised learning predicts human perception and misperception of gloss," Nature Human Behaviour, Nature, vol. 5(10), pages 1402-1417, October.
    13. Beau Sievers & Christopher Welker & Uri Hasson & Adam M. Kleinbaum & Thalia Wheatley, 2024. "Consensus-building conversation leads to neural alignment," Nature Communications, Nature, vol. 15(1), pages 1-15, December.
    14. Agustin Lage-Castellanos & Giancarlo Valente & Elia Formisano & Federico De Martino, 2019. "Methods for computing the maximum performance of computational models of fMRI responses," PLOS Computational Biology, Public Library of Science, vol. 15(3), pages 1-25, March.
    15. Lorenza Lucchi Basili & Pier Luigi Sacco, 2017. "Tie-Up Cycles in Long-Term Mating. Part II: Fictional Narratives and the Social Cognition of Mating," Challenges, MDPI, vol. 8(1), pages 1-60, February.
    16. Ming Bo Cai & Nicolas W Schuck & Jonathan W Pillow & Yael Niv, 2019. "Representational structure or task structure? Bias in neural representational similarity analysis and a Bayesian method for reducing bias," PLOS Computational Biology, Public Library of Science, vol. 15(5), pages 1-30, May.
    17. Timothy N Rubin & Oluwasanmi Koyejo & Krzysztof J Gorgolewski & Michael N Jones & Russell A Poldrack & Tal Yarkoni, 2017. "Decoding brain activity using a large-scale probabilistic functional-anatomical atlas of human cognition," PLOS Computational Biology, Public Library of Science, vol. 13(10), pages 1-24, October.
    18. Francesca Setti & Giacomo Handjaras & Davide Bottari & Andrea Leo & Matteo Diano & Valentina Bruno & Carla Tinti & Luca Cecchetti & Francesca Garbarini & Pietro Pietrini & Emiliano Ricciardi, 2023. "A modality-independent proto-organization of human multisensory areas," Nature Human Behaviour, Nature, vol. 7(3), pages 397-410, March.
    19. Desjardins, Christoph, 2021. "Don't be too SMART, but SAVE your goals: Proposal for a renewed goal-setting formula for Generation Y," Journal of Applied Leadership and Management, Hochschule Kempten - University of Applied Sciences, Professional School of Business & Technology, vol. 9, pages 73-87.
    20. Cai Wingfield & Li Su & Xunying Liu & Chao Zhang & Phil Woodland & Andrew Thwaites & Elisabeth Fonteneau & William D Marslen-Wilson, 2017. "Relating dynamic brain states to dynamic machine states: Human and machine solutions to the speech recognition problem," PLOS Computational Biology, Public Library of Science, vol. 13(9), pages 1-25, September.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nat:natcom:v:15:y:2024:i:1:d:10.1038_s41467-024-49173-5. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.nature.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.