IDEAS home Printed from https://ideas.repec.org/a/nat/natcom/v14y2023i1d10.1038_s41467-023-37139-y.html
   My bibliography  Save this article

AlphaFlow: autonomous discovery and optimization of multi-step chemistry using a self-driven fluidic lab guided by reinforcement learning

Author

Listed:
  • Amanda A. Volk

    (North Carolina State University)

  • Robert W. Epps

    (North Carolina State University)

  • Daniel T. Yonemoto

    (North Carolina State University)

  • Benjamin S. Masters

    (North Carolina State University)

  • Felix N. Castellano

    (North Carolina State University)

  • Kristofer G. Reyes

    (University at Buffalo)

  • Milad Abolhasani

    (North Carolina State University)

Abstract

Closed-loop, autonomous experimentation enables accelerated and material-efficient exploration of large reaction spaces without the need for user intervention. However, autonomous exploration of advanced materials with complex, multi-step processes and data sparse environments remains a challenge. In this work, we present AlphaFlow, a self-driven fluidic lab capable of autonomous discovery of complex multi-step chemistries. AlphaFlow uses reinforcement learning integrated with a modular microdroplet reactor capable of performing reaction steps with variable sequence, phase separation, washing, and continuous in-situ spectral monitoring. To demonstrate the power of reinforcement learning toward high dimensionality multi-step chemistries, we use AlphaFlow to discover and optimize synthetic routes for shell-growth of core-shell semiconductor nanoparticles, inspired by colloidal atomic layer deposition (cALD). Without prior knowledge of conventional cALD parameters, AlphaFlow successfully identified and optimized a novel multi-step reaction route, with up to 40 parameters, that outperformed conventional sequences. Through this work, we demonstrate the capabilities of closed-loop, reinforcement learning-guided systems in exploring and solving challenges in multi-step nanoparticle syntheses, while relying solely on in-house generated data from a miniaturized microfluidic platform. Further application of AlphaFlow in multi-step chemistries beyond cALD can lead to accelerated fundamental knowledge generation as well as synthetic route discoveries and optimization.

Suggested Citation

  • Amanda A. Volk & Robert W. Epps & Daniel T. Yonemoto & Benjamin S. Masters & Felix N. Castellano & Kristofer G. Reyes & Milad Abolhasani, 2023. "AlphaFlow: autonomous discovery and optimization of multi-step chemistry using a self-driven fluidic lab guided by reinforcement learning," Nature Communications, Nature, vol. 14(1), pages 1-16, December.
  • Handle: RePEc:nat:natcom:v:14:y:2023:i:1:d:10.1038_s41467-023-37139-y
    DOI: 10.1038/s41467-023-37139-y
    as

    Download full text from publisher

    File URL: https://www.nature.com/articles/s41467-023-37139-y
    File Function: Abstract
    Download Restriction: no

    File URL: https://libkey.io/10.1038/s41467-023-37139-y?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Jiagen Li & Junzi Li & Rulin Liu & Yuxiao Tu & Yiwen Li & Jiaji Cheng & Tingchao He & Xi Zhu, 2020. "Autonomous discovery of optically active chiral inorganic perovskite nanocrystals through an intelligent cloud lab," Nature Communications, Nature, vol. 11(1), pages 1-10, December.
    2. Qian Zhao & Abhijit Hazarika & Xihan Chen & Steve P. Harvey & Bryon W. Larson & Glenn R. Teeter & Jun Liu & Tao Song & Chuanxiao Xiao & Liam Shaw & Minghui Zhang & Guoran Li & Matthew C. Beard & Josep, 2019. "High efficiency perovskite quantum dot solar cells with charge separating heterostructure," Nature Communications, Nature, vol. 10(1), pages 1-8, December.
    3. Benjamin P. MacLeod & Fraser G. L. Parlane & Connor C. Rupnow & Kevan E. Dettelbach & Michael S. Elliott & Thomas D. Morrissey & Ted H. Haley & Oleksii Proskurin & Michael B. Rooney & Nina Taherimakhs, 2022. "A self-driving laboratory advances the Pareto front for material properties," Nature Communications, Nature, vol. 13(1), pages 1-10, December.
    4. David Silver & Aja Huang & Chris J. Maddison & Arthur Guez & Laurent Sifre & George van den Driessche & Julian Schrittwieser & Ioannis Antonoglou & Veda Panneershelvam & Marc Lanctot & Sander Dieleman, 2016. "Mastering the game of Go with deep neural networks and tree search," Nature, Nature, vol. 529(7587), pages 484-489, January.
    5. Benjamin Burger & Phillip M. Maffettone & Vladimir V. Gusev & Catherine M. Aitchison & Yang Bai & Xiaoyan Wang & Xiaobo Li & Ben M. Alston & Buyi Li & Rob Clowes & Nicola Rankin & Brandon Harris & Rei, 2020. "A mobile robotic chemist," Nature, Nature, vol. 583(7815), pages 237-241, July.
    6. Daniel Salley & Graham Keenan & Jonathan Grizou & Abhishek Sharma & Sergio Martín & Leroy Cronin, 2020. "A nanomaterials discovery robot for the Darwinian evolution of shape programmable gold nanoparticles," Nature Communications, Nature, vol. 11(1), pages 1-7, December.
    7. Charles R. Harris & K. Jarrod Millman & Stéfan J. Walt & Ralf Gommers & Pauli Virtanen & David Cournapeau & Eric Wieser & Julian Taylor & Sebastian Berg & Nathaniel J. Smith & Robert Kern & Matti Picu, 2020. "Array programming with NumPy," Nature, Nature, vol. 585(7825), pages 357-362, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Fernando Arteaga Cardona & Noopur Jain & Radian Popescu & Dmitry Busko & Eduard Madirov & Bernardo A. Arús & Dagmar Gerthsen & Annick Backer & Sara Bals & Oliver T. Bruns & Andriy Chmyrov & Sandra Aer, 2023. "Preventing cation intermixing enables 50% quantum yield in sub-15 nm short-wave infrared-emitting rare-earth based core-shell nanocrystals," Nature Communications, Nature, vol. 14(1), pages 1-14, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hyuk Jun Yoo & Kwan-Young Lee & Donghun Kim & Sang Soo Han, 2024. "OCTOPUS: operation control system for task optimization and job parallelization via a user-optimal scheduler," Nature Communications, Nature, vol. 15(1), pages 1-14, December.
    2. Benjamin P. MacLeod & Fraser G. L. Parlane & Connor C. Rupnow & Kevan E. Dettelbach & Michael S. Elliott & Thomas D. Morrissey & Ted H. Haley & Oleksii Proskurin & Michael B. Rooney & Nina Taherimakhs, 2022. "A self-driving laboratory advances the Pareto front for material properties," Nature Communications, Nature, vol. 13(1), pages 1-10, December.
    3. Adarsh Dave & Jared Mitchell & Sven Burke & Hongyi Lin & Jay Whitacre & Venkatasubramanian Viswanathan, 2022. "Autonomous optimization of non-aqueous Li-ion battery electrolytes via robotic experimentation and machine learning coupling," Nature Communications, Nature, vol. 13(1), pages 1-9, December.
    4. Amanda A. Volk & Milad Abolhasani, 2024. "Performance metrics to unleash the power of self-driving labs in chemistry and materials science," Nature Communications, Nature, vol. 15(1), pages 1-7, December.
    5. Justin J. Boutilier & Timothy C. Y. Chan, 2023. "Introducing and Integrating Machine Learning in an Operations Research Curriculum: An Application-Driven Course," INFORMS Transactions on Education, INFORMS, vol. 23(2), pages 64-83, January.
    6. Tan Wang & L. Jeff Hong, 2023. "Large-Scale Inventory Optimization: A Recurrent Neural Networks–Inspired Simulation Approach," INFORMS Journal on Computing, INFORMS, vol. 35(1), pages 196-215, January.
    7. Léon Faure & Bastien Mollet & Wolfram Liebermeister & Jean-Loup Faulon, 2023. "A neural-mechanistic hybrid approach improving the predictive power of genome-scale metabolic models," Nature Communications, Nature, vol. 14(1), pages 1-14, December.
    8. Claudia Quinteros-Cartaya & Guillermo Solorio-Magaña & Francisco Javier Núñez-Cornú & Felipe de Jesús Escalona-Alcázar & Diana Núñez, 2023. "Microearthquakes in the Guadalajara Metropolitan Zone, Mexico: evidence from buried active faults in Tesistán Valley, Zapopan," Natural Hazards: Journal of the International Society for the Prevention and Mitigation of Natural Hazards, Springer;International Society for the Prevention and Mitigation of Natural Hazards, vol. 116(3), pages 2797-2818, April.
    9. Tian Zhu & Merry H. Ma, 2022. "Deriving the Optimal Strategy for the Two Dice Pig Game via Reinforcement Learning," Stats, MDPI, vol. 5(3), pages 1-14, August.
    10. Xiaoyue Li & John M. Mulvey, 2023. "Optimal Portfolio Execution in a Regime-switching Market with Non-linear Impact Costs: Combining Dynamic Program and Neural Network," Papers 2306.08809, arXiv.org.
    11. Furqan Dar & Samuel R. Cohen & Diana M. Mitrea & Aaron H. Phillips & Gergely Nagy & Wellington C. Leite & Christopher B. Stanley & Jeong-Mo Choi & Richard W. Kriwacki & Rohit V. Pappu, 2024. "Biomolecular condensates form spatially inhomogeneous network fluids," Nature Communications, Nature, vol. 15(1), pages 1-17, December.
    12. Pedro Afonso Fernandes, 2024. "Forecasting with Neuro-Dynamic Programming," Papers 2404.03737, arXiv.org.
    13. Nathan Companez & Aldeida Aleti, 2016. "Can Monte-Carlo Tree Search learn to sacrifice?," Journal of Heuristics, Springer, vol. 22(6), pages 783-813, December.
    14. Yuchen Zhang & Wei Yang, 2022. "Breakthrough invention and problem complexity: Evidence from a quasi‐experiment," Strategic Management Journal, Wiley Blackwell, vol. 43(12), pages 2510-2544, December.
    15. Nina Tiel & Fabian Fopp & Philipp Brun & Johan Hoogen & Dirk Nikolaus Karger & Cecilia M. Casadei & Lisha Lyu & Devis Tuia & Niklaus E. Zimmermann & Thomas W. Crowther & Loïc Pellissier, 2024. "Regional uniqueness of tree species composition and response to forest loss and climate change," Nature Communications, Nature, vol. 15(1), pages 1-12, December.
    16. López Pérez, Mario & Mansilla Corona, Ricardo, 2022. "Ordinal synchronization and typical states in high-frequency digital markets," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 598(C).
    17. Yassine Chemingui & Adel Gastli & Omar Ellabban, 2020. "Reinforcement Learning-Based School Energy Management System," Energies, MDPI, vol. 13(23), pages 1-21, December.
    18. Zhewei Zhang & Youngjin Yoo & Kalle Lyytinen & Aron Lindberg, 2021. "The Unknowability of Autonomous Tools and the Liminal Experience of Their Use," Information Systems Research, INFORMS, vol. 32(4), pages 1192-1213, December.
    19. Yuhong Wang & Lei Chen & Hong Zhou & Xu Zhou & Zongsheng Zheng & Qi Zeng & Li Jiang & Liang Lu, 2021. "Flexible Transmission Network Expansion Planning Based on DQN Algorithm," Energies, MDPI, vol. 14(7), pages 1-21, April.
    20. JinHyo Joseph Yun & EuiSeob Jeong & Xiaofei Zhao & Sung Deuk Hahm & KyungHun Kim, 2019. "Collective Intelligence: An Emerging World in Open Innovation," Sustainability, MDPI, vol. 11(16), pages 1-15, August.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nat:natcom:v:14:y:2023:i:1:d:10.1038_s41467-023-37139-y. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.nature.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.