IDEAS home Printed from https://ideas.repec.org/a/nat/natcom/v16y2025i1d10.1038_s41467-025-56481-x.html
   My bibliography  Save this article

Transformer-generated atomic embeddings to enhance prediction accuracy of crystal properties with machine learning

Author

Listed:
  • Luozhijie Jin

    (Fudan University)

  • Zijian Du

    (Fudan University)

  • Le Shu

    (Fudan University)

  • Yan Cen

    (Fudan University)

  • Yuanfeng Xu

    (Shandong Jianzhu University)

  • Yongfeng Mei

    (Fudan University)

  • Hao Zhang

    (Fudan University
    Fudan University
    Fudan University)

Abstract

Accelerating the discovery of novel crystal materials by machine learning is crucial for advancing various technologies from clean energy to information processing. The machine-learning models for prediction of materials properties require embedding atomic information, while traditional methods have limited effectiveness in enhancing prediction accuracy. Here, we proposed an atomic embedding strategy called universal atomic embeddings (UAEs) for their broad applicability as atomic fingerprints, and generated the UAE tensors based on the proposed CrystalTransformer model. By performing experiments on widely-used materials database, our CrystalTransformer-based UAEs (ct-UAEs) are shown to accurately capture complex atomic features, leading to a 14% improvement in prediction accuracy on CGCNN and 18% on ALIGNN when using formation energies as the target, based on the Materials Project database. We also demonstrated the good transferability of ct-UAEs across various databases. Based on the clustering analysis for multi-task ct-UAEs, the elements in the periodic table can be categorized with reasonable connections between atomic features and targeted crystal properties. After applying ct-UAEs to predict formation energy in hybrid perovskites database, we realized an improvement in accuracy, with a 34% boost in MEGNET and 16% in CGCNN, showcasing their potential as atomic fingerprints to address the data scarcity challenges.

Suggested Citation

  • Luozhijie Jin & Zijian Du & Le Shu & Yan Cen & Yuanfeng Xu & Yongfeng Mei & Hao Zhang, 2025. "Transformer-generated atomic embeddings to enhance prediction accuracy of crystal properties with machine learning," Nature Communications, Nature, vol. 16(1), pages 1-11, December.
  • Handle: RePEc:nat:natcom:v:16:y:2025:i:1:d:10.1038_s41467-025-56481-x
    DOI: 10.1038/s41467-025-56481-x
    as

    Download full text from publisher

    File URL: https://www.nature.com/articles/s41467-025-56481-x
    File Function: Abstract
    Download Restriction: no

    File URL: https://libkey.io/10.1038/s41467-025-56481-x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Kathryn Tunyasuvunakool & Jonas Adler & Zachary Wu & Tim Green & Michal Zielinski & Augustin Žídek & Alex Bridgland & Andrew Cowie & Clemens Meyer & Agata Laydon & Sameer Velankar & Gerard J. Kleywegt, 2021. "Highly accurate protein structure prediction for the human proteome," Nature, Nature, vol. 596(7873), pages 590-596, August.
    2. Kristof T. Schütt & Farhad Arbabzadah & Stefan Chmiela & Klaus R. Müller & Alexandre Tkatchenko, 2017. "Quantum-chemical insights from deep tensor neural networks," Nature Communications, Nature, vol. 8(1), pages 1-8, April.
    3. Dipendra Jha & Kamal Choudhary & Francesca Tavazza & Wei-keng Liao & Alok Choudhary & Carelyn Campbell & Ankit Agrawal, 2019. "Enhancing materials property prediction by leveraging computational and experimental data using deep transfer learning," Nature Communications, Nature, vol. 10(1), pages 1-12, December.
    4. John Jumper & Richard Evans & Alexander Pritzel & Tim Green & Michael Figurnov & Olaf Ronneberger & Kathryn Tunyasuvunakool & Russ Bates & Augustin Žídek & Anna Potapenko & Alex Bridgland & Clemens Me, 2021. "Highly accurate protein structure prediction with AlphaFold," Nature, Nature, vol. 596(7873), pages 583-589, August.
    5. Keith T. Butler & Daniel W. Davies & Hugh Cartwright & Olexandr Isayev & Aron Walsh, 2018. "Machine learning for molecular and materials science," Nature, Nature, vol. 559(7715), pages 547-555, July.
    6. Alex Davies & Petar Veličković & Lars Buesing & Sam Blackwell & Daniel Zheng & Nenad Tomašev & Richard Tanburn & Peter Battaglia & Charles Blundell & András Juhász & Marc Lackenby & Geordie Williamson, 2021. "Advancing mathematics by guiding human intuition with AI," Nature, Nature, vol. 600(7887), pages 70-74, December.
    7. Shuaihua Lu & Qionghua Zhou & Yixin Ouyang & Yilv Guo & Qiang Li & Jinlan Wang, 2018. "Accelerated discovery of stable lead-free hybrid organic-inorganic perovskites via machine learning," Nature Communications, Nature, vol. 9(1), pages 1-8, December.
    8. Olexandr Isayev & Corey Oses & Cormac Toher & Eric Gossett & Stefano Curtarolo & Alexander Tropsha, 2017. "Universal fragment descriptors for predicting properties of inorganic crystals," Nature Communications, Nature, vol. 8(1), pages 1-12, August.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Xinyu Chen & Shuaihua Lu & Qian Chen & Qionghua Zhou & Jinlan Wang, 2024. "From bulk effective mass to 2D carrier mobility accurate prediction via adversarial transfer learning," Nature Communications, Nature, vol. 15(1), pages 1-9, December.
    2. Niklas W. A. Gebauer & Michael Gastegger & Stefaan S. P. Hessmann & Klaus-Robert Müller & Kristof T. Schütt, 2022. "Inverse design of 3d molecular structures with conditional generative neural networks," Nature Communications, Nature, vol. 13(1), pages 1-11, December.
    3. Hajkowicz, Stefan & Naughtin, Claire & Sanderson, Conrad & Schleiger, Emma & Karimi, Sarvnaz & Bratanova, Alexandra & Bednarz, Tomasz, 2022. "Artificial intelligence for science – adoption trends and future development pathways," MPRA Paper 115464, University Library of Munich, Germany.
    4. Zeyin Yan & Dacong Wei & Xin Li & Lung Wa Chung, 2024. "Accelerating reliable multiscale quantum refinement of protein–drug systems enabled by machine learning," Nature Communications, Nature, vol. 15(1), pages 1-12, December.
    5. Xin Li & Qunxi Zhu & Chengli Zhao & Xiaojun Duan & Bolin Zhao & Xue Zhang & Huanfei Ma & Jie Sun & Wei Lin, 2024. "Higher-order Granger reservoir computing: simultaneously achieving scalable complex structures inference and accurate dynamics prediction," Nature Communications, Nature, vol. 15(1), pages 1-13, December.
    6. Xuan-Kun Li & Jian-Xu Ma & Xiang-Yu Li & Jun-Jie Hu & Chuan-Yang Ding & Feng-Kai Han & Xiao-Min Guo & Xi Tan & Xian-Min Jin, 2024. "High-efficiency reinforcement learning with hybrid architecture photonic integrated circuit," Nature Communications, Nature, vol. 15(1), pages 1-10, December.
    7. Pierre Azoulay & Joshua Krieger & Abhishek Nagaraj, 2024. "Old Moats for New Models: Openness, Control, and Competition in Generative Artificial Intelligence," NBER Chapters, in: Entrepreneurship and Innovation Policy and the Economy, volume 4, pages 7-46, National Bureau of Economic Research, Inc.
    8. Jun-Yu Si & Yuan-Mei Chen & Ye-Hui Sun & Meng-Xue Gu & Mei-Ling Huang & Lu-Lu Shi & Xiao Yu & Xiao Yang & Qing Xiong & Cheng-Bao Ma & Peng Liu & Zheng-Li Shi & Huan Yan, 2024. "Sarbecovirus RBD indels and specific residues dictating multi-species ACE2 adaptiveness," Nature Communications, Nature, vol. 15(1), pages 1-15, December.
    9. Deyun Qiu & Jinxin V. Pei & James E. O. Rosling & Vandana Thathy & Dongdi Li & Yi Xue & John D. Tanner & Jocelyn Sietsma Penington & Yi Tong Vincent Aw & Jessica Yi Han Aw & Guoyue Xu & Abhai K. Tripa, 2022. "A G358S mutation in the Plasmodium falciparum Na+ pump PfATP4 confers clinically-relevant resistance to cipargamin," Nature Communications, Nature, vol. 13(1), pages 1-18, December.
    10. Shuo-Shuo Liu & Tian-Xia Jiang & Fan Bu & Ji-Lan Zhao & Guang-Fei Wang & Guo-Heng Yang & Jie-Yan Kong & Yun-Fan Qie & Pei Wen & Li-Bin Fan & Ning-Ning Li & Ning Gao & Xiao-Bo Qiu, 2024. "Molecular mechanisms underlying the BIRC6-mediated regulation of apoptosis and autophagy," Nature Communications, Nature, vol. 15(1), pages 1-16, December.
    11. Zhao-Shan Chen & Hsiang-Chi Huang & Xiangkun Wang & Karin Schön & Yane Jia & Michael Lebens & Danica F. Besavilla & Janarthan R. Murti & Yanhong Ji & Aishe A. Sarshad & Guohua Deng & Qiyun Zhu & David, 2025. "Influenza A Virus H7 nanobody recognizes a conserved immunodominant epitope on hemagglutinin head and confers heterosubtypic protection," Nature Communications, Nature, vol. 16(1), pages 1-17, December.
    12. Sourav Nayak & Thomas J. Peto & Michal Kucharski & Rupam Tripura & James J. Callery & Duong Tien Quang Huy & Mathieu Gendrot & Dysoley Lek & Ho Dang Trung Nghia & Rob W. Pluijm & Nguyen Dong & Le Than, 2024. "Population genomics and transcriptomics of Plasmodium falciparum in Cambodia and Vietnam uncover key components of the artemisinin resistance genetic background," Nature Communications, Nature, vol. 15(1), pages 1-17, December.
    13. Xiaoke Yang & Mingqi Zhu & Xue Lu & Yuxin Wang & Junyu Xiao, 2024. "Architecture and activation of human muscle phosphorylase kinase," Nature Communications, Nature, vol. 15(1), pages 1-14, December.
    14. Evangelos Katsamakas & Oleg V. Pavlov & Ryan Saklad, 2024. "Artificial intelligence and the transformation of higher education institutions," Papers 2402.08143, arXiv.org.
    15. Efren Garcia-Maldonado & Andrew D. Huber & Sergio C. Chai & Stanley Nithianantham & Yongtao Li & Jing Wu & Shyaron Poudel & Darcie J. Miller & Jayaraman Seetharaman & Taosheng Chen, 2024. "Chemical manipulation of an activation/inhibition switch in the nuclear receptor PXR," Nature Communications, Nature, vol. 15(1), pages 1-14, December.
    16. Kristy Rochon & Brianna L. Bauer & Nathaniel A. Roethler & Yuli Buckley & Chih-Chia Su & Wei Huang & Rajesh Ramachandran & Maria S. K. Stoll & Edward W. Yu & Derek J. Taylor & Jason A. Mears, 2024. "Structural basis for regulated assembly of the mitochondrial fission GTPase Drp1," Nature Communications, Nature, vol. 15(1), pages 1-10, December.
    17. Katherine A. Ray & Joshua D. Lutgens & Ramesh Bista & Jie Zhang & Ronak R. Desai & Melissa Hirsch & Takeshi Miyazawa & Antonio Cordova & Adrian T. Keatinge-Clay, 2024. "Assessing and harnessing updated polyketide synthase modules through combinatorial engineering," Nature Communications, Nature, vol. 15(1), pages 1-12, December.
    18. Fan Lu & Liang Zhu & Thomas Bromberger & Jun Yang & Qiannan Yang & Jianmin Liu & Edward F. Plow & Markus Moser & Jun Qin, 2022. "Mechanism of integrin activation by talin and its cooperation with kindlin," Nature Communications, Nature, vol. 13(1), pages 1-19, December.
    19. Zengyu Shao & Jiuwei Lu & Nelli Khudaverdyan & Jikui Song, 2024. "Multi-layered heterochromatin interaction as a switch for DIM2-mediated DNA methylation," Nature Communications, Nature, vol. 15(1), pages 1-18, December.
    20. Yudong Gao & Daichi Shonai & Matthew Trn & Jieqing Zhao & Erik J. Soderblom & S. Alexandra Garcia-Moreno & Charles A. Gersbach & William C. Wetsel & Geraldine Dawson & Dmitry Velmeshev & Yong-hui Jian, 2024. "Proximity analysis of native proteomes reveals phenotypic modifiers in a mouse model of autism and related neurodevelopmental conditions," Nature Communications, Nature, vol. 15(1), pages 1-18, December.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nat:natcom:v:16:y:2025:i:1:d:10.1038_s41467-025-56481-x. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.nature.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.