IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0059049.html
   My bibliography  Save this article

A Real-Time Brain-Machine Interface Combining Motor Target and Trajectory Intent Using an Optimal Feedback Control Design

Author

Listed:
  • Maryam M Shanechi
  • Ziv M Williams
  • Gregory W Wornell
  • Rollin C Hu
  • Marissa Powers
  • Emery N Brown

Abstract

Real-time brain-machine interfaces (BMI) have focused on either estimating the continuous movement trajectory or target intent. However, natural movement often incorporates both. Additionally, BMIs can be modeled as a feedback control system in which the subject modulates the neural activity to move the prosthetic device towards a desired target while receiving real-time sensory feedback of the state of the movement. We develop a novel real-time BMI using an optimal feedback control design that jointly estimates the movement target and trajectory of monkeys in two stages. First, the target is decoded from neural spiking activity before movement initiation. Second, the trajectory is decoded by combining the decoded target with the peri-movement spiking activity using an optimal feedback control design. This design exploits a recursive Bayesian decoder that uses an optimal feedback control model of the sensorimotor system to take into account the intended target location and the sensory feedback in its trajectory estimation from spiking activity. The real-time BMI processes the spiking activity directly using point process modeling. We implement the BMI in experiments consisting of an instructed-delay center-out task in which monkeys are presented with a target location on the screen during a delay period and then have to move a cursor to it without touching the incorrect targets. We show that the two-stage BMI performs more accurately than either stage alone. Correct target prediction can compensate for inaccurate trajectory estimation and vice versa. The optimal feedback control design also results in trajectories that are smoother and have lower estimation error. The two-stage decoder also performs better than linear regression approaches in offline cross-validation analyses. Our results demonstrate the advantage of a BMI design that jointly estimates the target and trajectory of movement and more closely mimics the sensorimotor control system.

Suggested Citation

  • Maryam M Shanechi & Ziv M Williams & Gregory W Wornell & Rollin C Hu & Marissa Powers & Emery N Brown, 2013. "A Real-Time Brain-Machine Interface Combining Motor Target and Trajectory Intent Using an Optimal Feedback Control Design," PLOS ONE, Public Library of Science, vol. 8(4), pages 1-16, April.
  • Handle: RePEc:plo:pone00:0059049
    DOI: 10.1371/journal.pone.0059049
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0059049
    Download Restriction: no

    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0059049&type=printable
    Download Restriction: no

    File URL: https://libkey.io/10.1371/journal.pone.0059049?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Zheng Li & Joseph E O'Doherty & Timothy L Hanson & Mikhail A Lebedev & Craig S Henriquez & Miguel A L Nicolelis, 2009. "Unscented Kalman Filter for Brain-Machine Interfaces," PLOS ONE, Public Library of Science, vol. 4(7), pages 1-18, July.
    2. Johan Wessberg & Christopher R. Stambaugh & Jerald D. Kralik & Pamela D. Beck & Mark Laubach & John K. Chapin & Jung Kim & S. James Biggs & Mandayam A. Srinivasan & Miguel A. L. Nicolelis, 2000. "Real-time prediction of hand trajectory by ensembles of cortical neurons in primates," Nature, Nature, vol. 408(6810), pages 361-365, November.
    3. Mijail D. Serruya & Nicholas G. Hatsopoulos & Liam Paninski & Matthew R. Fellows & John P. Donoghue, 2002. "Instant neural control of a movement signal," Nature, Nature, vol. 416(6877), pages 141-142, March.
    4. Christopher A. Buneo & Murray R. Jarvis & Aaron P. Batista & Richard A. Andersen, 2002. "Direct visuomotor transformations for reaching," Nature, Nature, vol. 416(6881), pages 632-636, April.
    5. Joseph E. O’Doherty & Mikhail A. Lebedev & Peter J. Ifft & Katie Z. Zhuang & Solaiman Shokur & Hannes Bleuler & Miguel A. L. Nicolelis, 2011. "Active tactile exploration using a brain–machine–brain interface," Nature, Nature, vol. 479(7372), pages 228-231, November.
    6. L. H. Snyder & A. P. Batista & R. A. Andersen, 1997. "Coding of intention in the posterior parietal cortex," Nature, Nature, vol. 386(6621), pages 167-170, March.
    7. Rong Chen & Jun S. Liu, 2000. "Mixture Kalman filters," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 62(3), pages 493-508.
    8. Gopal Santhanam & Stephen I. Ryu & Byron M. Yu & Afsheen Afshar & Krishna V. Shenoy, 2006. "A high-performance brain–computer interface," Nature, Nature, vol. 442(7099), pages 195-198, July.
    9. Meel Velliste & Sagi Perel & M. Chance Spalding & Andrew S. Whitford & Andrew B. Schwartz, 2008. "Cortical control of a prosthetic arm for self-feeding," Nature, Nature, vol. 453(7198), pages 1098-1101, June.
    10. Leigh R. Hochberg & Mijail D. Serruya & Gerhard M. Friehs & Jon A. Mukand & Maryam Saleh & Abraham H. Caplan & Almut Branner & David Chen & Richard D. Penn & John P. Donoghue, 2006. "Neuronal ensemble control of prosthetic devices by a human with tetraplegia," Nature, Nature, vol. 442(7099), pages 164-171, July.
    11. Chet T. Moritz & Steve I. Perlmutter & Eberhard E. Fetz, 2008. "Direct control of paralysed muscles by cortical neurons," Nature, Nature, vol. 456(7222), pages 639-642, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yasuhiko Nakanishi & Takufumi Yanagisawa & Duk Shin & Ryohei Fukuma & Chao Chen & Hiroyuki Kambara & Natsue Yoshimura & Masayuki Hirata & Toshiki Yoshimine & Yasuharu Koike, 2013. "Prediction of Three-Dimensional Arm Trajectories Based on ECoG Signals Recorded from Human Sensorimotor Cortex," PLOS ONE, Public Library of Science, vol. 8(8), pages 1-9, August.
    2. Zheng Li & Joseph E O'Doherty & Timothy L Hanson & Mikhail A Lebedev & Craig S Henriquez & Miguel A L Nicolelis, 2009. "Unscented Kalman Filter for Brain-Machine Interfaces," PLOS ONE, Public Library of Science, vol. 4(7), pages 1-18, July.
    3. Josh Merel & David Carlson & Liam Paninski & John P Cunningham, 2016. "Neuroprosthetic Decoder Training as Imitation Learning," PLOS Computational Biology, Public Library of Science, vol. 12(5), pages 1-24, May.
    4. Alessandro Vato & Marianna Semprini & Emma Maggiolini & Francois D Szymanski & Luciano Fadiga & Stefano Panzeri & Ferdinando A Mussa-Ivaldi, 2012. "Shaping the Dynamics of a Bidirectional Neural Interface," PLOS Computational Biology, Public Library of Science, vol. 8(7), pages 1-15, July.
    5. Nuri F Ince & Rahul Gupta & Sami Arica & Ahmed H Tewfik & James Ashe & Giuseppe Pellizzer, 2010. "High Accuracy Decoding of Movement Target Direction in Non-Human Primates Based on Common Spatial Patterns of Local Field Potentials," PLOS ONE, Public Library of Science, vol. 5(12), pages 1-11, December.
    6. Burkhart, Michael C., 2019. "A Discriminative Approach to Bayesian Filtering with Applications to Human Neural Decoding," Thesis Commons 4j3fu_v1, Center for Open Science.
    7. Hong Gi Yeom & June Sic Kim & Chun Kee Chung, 2014. "High-Accuracy Brain-Machine Interfaces Using Feedback Information," PLOS ONE, Public Library of Science, vol. 9(7), pages 1-7, July.
    8. Andrey Eliseyev & Tetiana Aksenova, 2016. "Penalized Multi-Way Partial Least Squares for Smooth Trajectory Decoding from Electrocorticographic (ECoG) Recording," PLOS ONE, Public Library of Science, vol. 11(5), pages 1-19, May.
    9. Shinsuke Koyama & Uri Eden & Emery Brown & Robert Kass, 2010. "Bayesian decoding of neural spike trains," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 62(1), pages 37-59, February.
    10. Bahareh Taghizadeh & Ole Fortmann & Alexander Gail, 2024. "Position- and scale-invariant object-centered spatial localization in monkey frontoparietal cortex dynamically adapts to cognitive demand," Nature Communications, Nature, vol. 15(1), pages 1-15, December.
    11. Tomislav Milekovic & Tonio Ball & Andreas Schulze-Bonhage & Ad Aertsen & Carsten Mehring, 2013. "Detection of Error Related Neuronal Responses Recorded by Electrocorticography in Humans during Continuous Movements," PLOS ONE, Public Library of Science, vol. 8(2), pages 1-20, February.
    12. Pavlov, A.N. & Grishina, D.S. & Runnova, A.E. & Maksimenko, V.A. & Pavlova, O.N. & Shchukovsky, N.V. & Hramov, A.E. & Kurths, J., 2019. "Recognition of electroencephalographic patterns related to human movements or mental intentions with multiresolution analysis," Chaos, Solitons & Fractals, Elsevier, vol. 126(C), pages 230-235.
    13. Andrés Úbeda & Enrique Hortal & Eduardo Iáñez & Carlos Perez-Vidal & Jose M Azorín, 2015. "Assessing Movement Factors in Upper Limb Kinematics Decoding from EEG Signals," PLOS ONE, Public Library of Science, vol. 10(5), pages 1-12, May.
    14. Leif Anders Thorsrud, 2016. "Nowcasting using news topics Big Data versus big bank," Working Papers No 6/2016, Centre for Applied Macro- and Petroleum economics (CAMP), BI Norwegian Business School.
    15. Mengheng Li & Siem Jan (S.J.) Koopman, 2018. "Unobserved Components with Stochastic Volatility in U.S. Inflation: Estimation and Signal Extraction," Tinbergen Institute Discussion Papers 18-027/III, Tinbergen Institute.
    16. James M. Nason & Gregor W. Smith, 2021. "Measuring the slowly evolving trend in US inflation with professional forecasts," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 36(1), pages 1-17, January.
    17. Koop, Gary & Korobilis, Dimitris, 2011. "UK macroeconomic forecasting with many predictors: Which models forecast best and when do they do so?," Economic Modelling, Elsevier, vol. 28(5), pages 2307-2318, September.
    18. Larissa Albantakis & Francesca M Branzi & Albert Costa & Gustavo Deco, 2012. "A Multiple-Choice Task with Changes of Mind," PLOS ONE, Public Library of Science, vol. 7(8), pages 1-15, August.
    19. Siddhartha Chib & Minchul Shin & Fei Tan, 2023. "DSGE-SVt: An Econometric Toolkit for High-Dimensional DSGE Models with SV and t Errors," Computational Economics, Springer;Society for Computational Economics, vol. 61(1), pages 69-111, January.
    20. Linlin Li & Shufang Zhao & Wenhao Ran & Zhexin Li & Yongxu Yan & Bowen Zhong & Zheng Lou & Lili Wang & Guozhen Shen, 2022. "Dual sensing signal decoupling based on tellurium anisotropy for VR interaction and neuro-reflex system application," Nature Communications, Nature, vol. 13(1), pages 1-12, December.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0059049. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.