Paul Nuyujukian

All Publications

• A High-Performance Keyboard Neural Prosthesis Enabled by Task Optimization IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING Nuyujukian, P., Fan, J. M., Kao, J. C., Ryu, S. I., Shenoy, K. V. 2015; 62 (1): 21-29

  Abstract

  Communication neural prostheses are an emerging class of medical devices that aim to restore efficient communication to people suffering from paralysis. These systems rely on an interface with the user, either via the use of a continuouslymoving cursor (e.g., mouse) or the discrete selection of symbols (e.g., keyboard). In developing these interfaces, many design choices have a significant impact on the performance of the system. The objective of this study was to explore the design choices of a continuously-moving cursor neural prosthesis and optimize the interface to maximize information theoretic performance. We swept interface parameters of two keyboard-like tasks to find task and subject specific optimal parameters as measured by achieved bitrate using two rhesus macaques implanted with multielectrode arrays. In this report, we present the highest performing free-paced neural prosthesis under any recording modality with sustainable communication rates of up to 3.5 bits per second (bps). These findings demonstrate that meaningful high performance can be achieved using an intracortical neural prosthesis, and that, when optimized, these systems may be appropriate for use as communication devices for those with physical disabilities.

  View details for DOI 10.1109/TBME.2014.2354697

  View details for Web of Science ID 000346765500003

• Performance sustaining intracortical neural prostheses. Journal of neural engineering Nuyujukian, P., Kao, J. C., Fan, J. M., Stavisky, S. D., Ryu, S. I., Shenoy, K. V. 2014; 11 (6): 066003-?

  Abstract

  Objective. Neural prostheses, or brain-machine interfaces, aim to restore efficient communication and movement ability to those suffering from paralysis. A major challenge these systems face is robust performance, particularly with aging signal sources. The aim in this study was to develop a neural prosthesis that could sustain high performance in spite of signal instability while still minimizing retraining time. Approach. We trained two rhesus macaques implanted with intracortical microelectrode arrays 1-4 years prior to this study to acquire targets with a neurally-controlled cursor. We measured their performance via achieved bitrate (bits per second, bps). This task was repeated over contiguous days to evaluate the sustained performance across time. Main results. We found that in the monkey with a younger (i.e., two year old) implant and better signal quality, a fixed decoder could sustain performance for a month at a rate of 4 bps, the highest achieved communication rate reported to date. This fixed decoder was evaluated across 22 months and experienced a performance decline at a rate of 0.24 bps yr(-1). In the monkey with the older (i.e., 3.5 year old) implant and poorer signal quality, a fixed decoder could not sustain performance for more than a few days. Nevertheless, performance in this monkey was maintained for two weeks without requiring additional online retraining time by utilizing prior days' experimental data. Upon analysis of the changes in channel tuning, we found that this stability appeared partially attributable to the cancelling-out of neural tuning fluctuations when projected to two-dimensional cursor movements. Significance. The findings in this study (1) document the highest-performing communication neural prosthesis in monkeys, (2) confirm and extend prior reports of the stability of fixed decoders, and (3) demonstrate a protocol for system stability under conditions where fixed decoders would otherwise fail. These improvements to decoder stability are important for minimizing training time and should make neural prostheses more practical to use.

  View details for DOI 10.1088/1741-2560/11/6/066003

  View details for PubMedID 25307561

• A high-performance neural prosthesis enabled by control algorithm design NATURE NEUROSCIENCE Gilja, V., Nuyujukian, P., Chestek, C. A., Cunningham, J. P., Yu, B. M., Fan, J. M., Churchland, M. M., Kaufman, M. T., Kao, J. C., Ryu, S. I., Shenoy, K. V. 2012; 15 (12): 1752-1757

  Abstract

  Neural prostheses translate neural activity from the brain into control signals for guiding prosthetic devices, such as computer cursors and robotic limbs, and thus offer individuals with disabilities greater interaction with the world. However, relatively low performance remains a critical barrier to successful clinical translation; current neural prostheses are considerably slower, with less accurate control, than the native arm. Here we present a new control algorithm, the recalibrated feedback intention-trained Kalman filter (ReFIT-KF) that incorporates assumptions about the nature of closed-loop neural prosthetic control. When tested in rhesus monkeys implanted with motor cortical electrode arrays, the ReFIT-KF algorithm outperformed existing neural prosthetic algorithms in all measured domains and halved target acquisition time. This control algorithm permits sustained, uninterrupted use for hours and generalizes to more challenging tasks without retraining. Using this algorithm, we demonstrate repeatable high performance for years after implantation in two monkeys, thereby increasing the clinical viability of neural prostheses.

  View details for DOI 10.1038/nn.3265

  View details for Web of Science ID 000311706700023

  View details for PubMedID 23160043

• Clinical translation of a high-performance neural prosthesis. Nature medicine Gilja, V., Pandarinath, C., Blabe, C. H., Nuyujukian, P., Simeral, J. D., Sarma, A. A., Sorice, B. L., Perge, J. A., Jarosiewicz, B., Hochberg, L. R., Shenoy, K. V., Henderson, J. M. 2015; 21 (10): 1142-1145

  Abstract

  Neural prostheses have the potential to improve the quality of life of individuals with paralysis by directly mapping neural activity to limb- and computer-control signals. We translated a neural prosthetic system previously developed in animal model studies for use by two individuals with amyotrophic lateral sclerosis who had intracortical microelectrode arrays placed in motor cortex. Measured more than 1 year after implant, the neural cursor-control system showed the highest published performance achieved by a person to date, more than double that of previous pilot clinical trial participants.

  View details for DOI 10.1038/nm.3953

  View details for PubMedID 26413781

• Single-trial dynamics of motor cortex and their applications to brain-machine interfaces NATURE COMMUNICATIONS Kao, J. C., Nuyujukian, P., Ryu, S. I., Churchland, M. M., Cunningham, J. P., Shenoy, K. V. 2015; 6

  View details for DOI 10.1038/ncomms8759

  View details for Web of Science ID 000358858800003

• Neural population dynamics in human motor cortex during movements in people with ALS ELIFE Pandarinath, C., Gilja, V., Blabe, C. H., Nuyujukian, P., Sarma, A. A., Sorice, B. L., Eskandar, E. N., Hochberg, L. R., Henderson, J. M., Shenoy, K. V. 2015; 4

  View details for DOI 10.7554/eLife.07436

  View details for Web of Science ID 000356720100001

• A high performing brain-machine interface driven by low-frequency local field potentials alone and together with spikes. Journal of neural engineering Stavisky, S. D., Kao, J. C., Nuyujukian, P., Ryu, S. I., Shenoy, K. V. 2015; 12 (3): 036009-?

  Abstract

  Brain-machine interfaces (BMIs) seek to enable people with movement disabilities to directly control prosthetic systems with their neural activity. Current high performance BMIs are driven by action potentials (spikes), but access to this signal often diminishes as sensors degrade over time. Decoding local field potentials (LFPs) as an alternative or complementary BMI control signal may improve performance when there is a paucity of spike signals. To date only a small handful of LFP decoding methods have been tested online; there remains a need to test different LFP decoding approaches and improve LFP-driven performance. There has also not been a reported demonstration of a hybrid BMI that decodes kinematics from both LFP and spikes. Here we first evaluate a BMI driven by the local motor potential (LMP), a low-pass filtered time-domain LFP amplitude feature. We then combine decoding of both LMP and spikes to implement a hybrid BMI.Spikes and LFP were recorded from two macaques implanted with multielectrode arrays in primary and premotor cortex while they performed a reaching task. We then evaluated closed-loop BMI control using biomimetic decoders driven by LMP, spikes, or both signals together.LMP decoding enabled quick and accurate cursor control which surpassed previously reported LFP BMI performance. Hybrid decoding of both spikes and LMP improved performance when spikes signal quality was mediocre to poor.These findings show that LMP is an effective BMI control signal which requires minimal power to extract and can substitute for or augment impoverished spikes signals. Use of this signal may lengthen the useful lifespan of BMIs and is therefore an important step towards clinically viable BMIs.

  View details for DOI 10.1088/1741-2560/12/3/036009

  View details for PubMedID 25946198

• A high performing brain-machine interface driven by low-frequency local field potentials alone and together with spikes JOURNAL OF NEURAL ENGINEERING Stavisky, S. D., Kao, J. C., Nuyujukian, P., Ryu, S. I., Shenoy, K. V. 2015; 12 (3)

  View details for DOI 10.1088/1741-2560/12/3/036009

  View details for Web of Science ID 000354998600010

• Comparison of spike sorting and thresholding of voltage waveforms for intracortical brain-machine interface performance. Journal of neural engineering Christie, B. P., Tat, D. M., Irwin, Z. T., Gilja, V., Nuyujukian, P., Foster, J. D., Ryu, S. I., Shenoy, K. V., Thompson, D. E., Chestek, C. A. 2015; 12 (1): 016009-?

  Abstract

  Objective. For intracortical brain-machine interfaces (BMIs), action potential voltage waveforms are often sorted to separate out individual neurons. If these neurons contain independent tuning information, this process could increase BMI performance. However, the sorting of action potentials ('spikes') requires high sampling rates and is computationally expensive. To explicitly define the difference between spike sorting and alternative methods, we quantified BMI decoder performance when using threshold-crossing events versus sorted action potentials. Approach. We used data sets from 58 experimental sessions from two rhesus macaques implanted with Utah arrays. Data were recorded while the animals performed a center-out reaching task with seven different angles. For spike sorting, neural signals were sorted into individual units by using a mixture of Gaussians to cluster the first four principal components of the waveforms. For thresholding events, spikes that simply crossed a set threshold were retained. We decoded the data offline using both a Naïve Bayes classifier for reaching direction and a linear regression to evaluate hand position. Main results. We found the highest performance for thresholding when placing a threshold between -3 and -4.5 × Vrms. Spike sorted data outperformed thresholded data for one animal but not the other. The mean Naïve Bayes classification accuracy for sorted data was 88.5% and changed by 5% on average when data were thresholded. The mean correlation coefficient for sorted data was 0.92, and changed by 0.015 on average when thresholded. Significance. For prosthetics applications, these results imply that when thresholding is used instead of spike sorting, only a small amount of performance may be lost. The utilization of threshold-crossing events may significantly extend the lifetime of a device because these events are often still detectable once single neurons are no longer isolated.

  View details for DOI 10.1088/1741-2560/12/1/016009

  View details for PubMedID 25504690

• Comparison of spike sorting and thresholding of voltage waveforms for intracortical brain-machine interface performance JOURNAL OF NEURAL ENGINEERING Christie, B. P., Tat, D. M., Irwin, Z. T., Gilja, V., Nuyujukian, P., Foster, J. D., Ryu, S. I., Shenoy, K. V., Thompson, D. E., Chestek, C. A. 2015; 12 (1)

  View details for DOI 10.1088/1741-2560/12/1/016009

  View details for Web of Science ID 000348762500011

• Single-trial dynamics of motor cortex and their applications to brain-machine interfaces. Nature communications Kao, J. C., Nuyujukian, P., Ryu, S. I., Churchland, M. M., Cunningham, J. P., Shenoy, K. V. 2015; 6: 7759-?

  Abstract

  Increasing evidence suggests that neural population responses have their own internal drive, or dynamics, that describe how the neural population evolves through time. An important prediction of neural dynamical models is that previously observed neural activity is informative of noisy yet-to-be-observed activity on single-trials, and may thus have a denoising effect. To investigate this prediction, we built and characterized dynamical models of single-trial motor cortical activity. We find these models capture salient dynamical features of the neural population and are informative of future neural activity on single trials. To assess how neural dynamics may beneficially denoise single-trial neural activity, we incorporate neural dynamics into a brain-machine interface (BMI). In online experiments, we find that a neural dynamical BMI achieves substantially higher performance than its non-dynamical counterpart. These results provide evidence that neural dynamics beneficially inform the temporal evolution of neural activity on single trials and may directly impact the performance of BMIs.

  View details for DOI 10.1038/ncomms8759

  View details for PubMedID 26220660

• Neural population dynamics in human motor cortex during movements in people with ALS. eLife Pandarinath, C., Gilja, V., Blabe, C. H., Nuyujukian, P., Sarma, A. A., Sorice, B. L., Eskandar, E. N., Hochberg, L. R., Henderson, J. M., Shenoy, K. V. 2015; 4

  Abstract

  The prevailing view of motor cortex holds that motor cortical neural activity represents muscle or movement parameters. However, recent studies in non-human primates have shown that neural activity does not simply represent muscle or movement parameters; instead, its temporal structure is well-described by a dynamical system where activity during movement evolves lawfully from an initial pre-movement state. In this study, we analyze neuronal ensemble activity in motor cortex in two clinical trial participants diagnosed with Amyotrophic Lateral Sclerosis (ALS). We find that activity in human motor cortex has similar dynamical structure to that of non-human primates, indicating that human motor cortex contains a similar underlying dynamical system for movement generation.

  View details for DOI 10.7554/eLife.07436

  View details for PubMedID 26099302

• Performance sustaining intracortical neural prostheses JOURNAL OF NEURAL ENGINEERING Nuyujukian, P., Kao, J. C., Fan, J. M., Stavisky, S. D., Ryu, S. I., Shenoy, K. V. 2014; 11 (6)

  View details for DOI 10.1088/1741-2560/11/6/066003

  View details for Web of Science ID 000345647700009

• A freely-moving monkey treadmill model. Journal of neural engineering D Foster, J., Nuyujukian, P., Freifeld, O., Gao, H., Walker, R., I Ryu, S., H Meng, T., Murmann, B., J Black, M., V Shenoy, K. 2014; 11 (4): 046020-?

  Abstract

  Objective. Motor neuroscience and brain-machine interface (BMI) design is based on examining how the brain controls voluntary movement, typically by recording neural activity and behavior from animal models. Recording technologies used with these animal models have traditionally limited the range of behaviors that can be studied, and thus the generality of science and engineering research. We aim to design a freely-moving animal model using neural and behavioral recording technologies that do not constrain movement. Approach. We have established a freely-moving rhesus monkey model employing technology that transmits neural activity from an intracortical array using a head-mounted device and records behavior through computer vision using markerless motion capture. We demonstrate the flexibility and utility of this new monkey model, including the first recordings from motor cortex while rhesus monkeys walk quadrupedally on a treadmill. Main results. Using this monkey model, we show that multi-unit threshold-crossing neural activity encodes the phase of walking and that the average firing rate of the threshold crossings covaries with the speed of individual steps. On a population level, we find that neural state-space trajectories of walking at different speeds have similar rotational dynamics in some dimensions that evolve at the step rate of walking, yet robustly separate by speed in other state-space dimensions. Significance. Freely-moving animal models may allow neuroscientists to examine a wider range of behaviors and can provide a flexible experimental paradigm for examining the neural mechanisms that underlie movement generation across behaviors and environments. For BMIs, freely-moving animal models have the potential to aid prosthetic design by examining how neural encoding changes with posture, environment and other real-world context changes. Understanding this new realm of behavior in more naturalistic settings is essential for overall progress of basic motor neuroscience and for the successful translation of BMIs to people with paralysis.

  View details for DOI 10.1088/1741-2560/11/4/046020

  View details for PubMedID 24995476

• Information Systems Opportunities in Brain-Machine Interface Decoders PROCEEDINGS OF THE IEEE Kao, J. C., Stavisky, S. D., Sussillo, D., Nuyujukian, P., Shenoy, K. V. 2014; 102 (5): 666-682

  View details for DOI 10.1109/JPROC.2014.2307357

  View details for Web of Science ID 000336915700004

• Self-recalibrating classifiers for intracortical brain-computer interfaces. Journal of neural engineering Bishop, W., Chestek, C. C., Gilja, V., Nuyujukian, P., Foster, J. D., Ryu, S. I., Shenoy, K. V., Yu, B. M. 2014; 11 (2): 026001-?

  Abstract

  Objective. Intracortical brain-computer interface (BCI) decoders are typically retrained daily to maintain stable performance. Self-recalibrating decoders aim to remove the burden this may present in the clinic by training themselves autonomously during normal use but have only been developed for continuous control. Here we address the problem for discrete decoding (classifiers). Approach. We recorded threshold crossings from 96-electrode arrays implanted in the motor cortex of two rhesus macaques performing center-out reaches in 7 directions over 41 and 36 separate days spanning 48 and 58 days in total for offline analysis. Main results. We show that for the purposes of developing a self-recalibrating classifier, tuning parameters can be considered as fixed within days and that parameters on the same electrode move up and down together between days. Further, drift is constrained across time, which is reflected in the performance of a standard classifier which does not progressively worsen if it is not retrained daily, though overall performance is reduced by more than 10% compared to a daily retrained classifier. Two novel self-recalibrating classifiers produce a [Formula: see text] increase in classification accuracy over that achieved by the non-retrained classifier to nearly recover the performance of the daily retrained classifier. Significance. We believe that the development of classifiers that require no daily retraining will accelerate the clinical translation of BCI systems. Future work should test these results in a closed-loop setting.

  View details for DOI 10.1088/1741-2560/11/2/026001

  View details for PubMedID 24503597

• Self-recalibrating classifiers for intracortical brain-computer interfaces JOURNAL OF NEURAL ENGINEERING Bishop, W., Chestek, C. C., Gilja, V., Nuyujukian, P., Foster, J. D., Ryu, S. I., Shenoy, K. V., Yu, B. M. 2014; 11 (2)

  View details for DOI 10.1088/1741-2560/11/2/026001

  View details for Web of Science ID 000333419400006

• Intention estimation in brain-machine interfaces. Journal of neural engineering Fan, J. M., Nuyujukian, P., Kao, J. C., Chestek, C. A., Ryu, S. I., Shenoy, K. V. 2014; 11 (1): 016004-?

  Abstract

  The objective of this work was to quantitatively investigate the mechanisms underlying the performance gains of the recently reported 'recalibrated feedback intention-trained Kalman Filter' (ReFIT-KF).This was accomplished by designing variants of the ReFIT-KF algorithm and evaluating training and online data to understand the neural basis of this improvement. We focused on assessing the contribution of two training set innovations of the ReFIT-KF algorithm: intention estimation and the two-stage training paradigm.Within the two-stage training paradigm, we found that intention estimation independently increased target acquisition rates by 37% and 59%, respectively, across two monkeys implanted with multiunit intracortical arrays. Intention estimation improved performance by enhancing the tuning properties and the mutual information between the kinematic and neural training data. Furthermore, intention estimation led to fewer shifts in channel tuning between the training set and online control, suggesting that less adaptation was required during online control. Retraining the decoder with online BMI training data also reduced shifts in tuning, suggesting a benefit of training a decoder in the same behavioral context; however, retraining also led to slower online decode velocities. Finally, we demonstrated that one- and two-stage training paradigms performed comparably when intention estimation is applied.These findings highlight the utility of intention estimation in reducing the need for adaptive strategies and improving the online performance of BMIs, helping to guide future BMI design decisions.

  View details for PubMedID 24654266

• Intention estimation in brain-machine interfaces JOURNAL OF NEURAL ENGINEERING Fan, J. M., Nuyujukian, P., Kao, J. C., Chestek, C. A., Ryu, S. I., Shenoy, K. V. 2014; 11 (1)

  View details for DOI 10.1088/1741-2560/11/1/016004

  View details for Web of Science ID 000333165000004

• Hybrid decoding of both spikes and low-frequency local field potentials for brain-machine interfaces. Conference proceedings : ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual Conference Stavisky, S. D., Kao, J. C., Nuyujukian, P., Ryu, S. I., Shenoy, K. V. 2014; 2014: 3041-3044

  Abstract

  The best-performing brain-machine interfaces (BMIs) to date decode movement intention from intracortically recorded spikes, but these signals may be lost over time. A way to increase the useful lifespan of BMIs is to make more comprehensive use of available neural signals. Recent studies have demonstrated that the local field potential (LFP), a potentially more robust signal, can also be used to control a BMI. However, LFP-driven performance has fallen short of the best spikes-driven performance. Here we report a biomimetic BMI driven by low-frequency LFP that enabled a rhesus monkey to acquire and hold randomly placed targets with 99% success rate. Although LFP-driven performance was still worse than when decoding spikes, to the best of our knowledge this represents the highest-performing LFP-based BMI. We also demonstrate a new hybrid BMI that decodes cursor velocity using both spikes and LFP. This hybrid decoder improved performance over spikes-only decoding. Our results suggest that LFP can complement spikes when spikes are available or provide an alternative control signal if spikes are absent.

  View details for DOI 10.1109/EMBC.2014.6944264

  View details for PubMedID 25570632

• Investigating the role of firing-rate normalization and dimensionality reduction in brain-machine interface robustness. Conference proceedings : ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Conference Kao, J. C., Nuyujukian, P., Stavisky, S., Ryu, S. I., Ganguli, S., Shenoy, K. V. 2013; 2013: 293-298

  Abstract

  The intraday robustness of brain-machine interfaces (BMIs) is important to their clinical viability. In particular, BMIs must be robust to intraday perturbations in neuron firing rates, which may arise from several factors including recording loss and external noise. Using a state-of-the-art decode algorithm, the Recalibrated Feedback Intention Trained Kalman filter (ReFIT-KF) [1] we introduce two novel modifications: (1) a normalization of the firing rates, and (2) a reduction of the dimensionality of the data via principal component analysis (PCA). We demonstrate in online studies that a ReFIT-KF equipped with normalization and PCA (NPC-ReFIT-KF) (1) achieves comparable performance to a standard ReFIT-KF when at least 60% of the neural variance is captured, and (2) is more robust to the undetected loss of channels. We present intuition as to how both modifications may increase the robustness of BMIs, and investigate the contribution of each modification to robustness. These advances, which lead to a decoder achieving state-of-the-art performance with improved robustness, are important for the clinical viability of BMI systems.

  View details for DOI 10.1109/EMBC.2013.6609495

  View details for PubMedID 24109682

• Design and validation of a real-time spiking-neural-network decoder for brain-machine interfaces. Journal of neural engineering Dethier, J., Nuyujukian, P., Ryu, S. I., Shenoy, K. V., Boahen, K. 2013; 10 (3): 036008-?

  Abstract

  Objective. Cortically-controlled motor prostheses aim to restore functions lost to neurological disease and injury. Several proof of concept demonstrations have shown encouraging results, but barriers to clinical translation still remain. In particular, intracortical prostheses must satisfy stringent power dissipation constraints so as not to damage cortex. Approach. One possible solution is to use ultra-low power neuromorphic chips to decode neural signals for these intracortical implants. The first step is to explore in simulation the feasibility of translating decoding algorithms for brain-machine interface (BMI) applications into spiking neural networks (SNNs). Main results. Here we demonstrate the validity of the approach by implementing an existing Kalman-filter-based decoder in a simulated SNN using the Neural Engineering Framework (NEF), a general method for mapping control algorithms onto SNNs. To measure this system's robustness and generalization, we tested it online in closed-loop BMI experiments with two rhesus monkeys. Across both monkeys, a Kalman filter implemented using a 2000-neuron SNN has comparable performance to that of a Kalman filter implemented using standard floating point techniques. Significance. These results demonstrate the tractability of SNN implementations of statistical signal processing algorithms on different monkeys and for several tasks, suggesting that a SNN decoder, implemented on a neuromorphic chip, may be a feasible computational platform for low-power fully-implanted prostheses. The validation of this closed-loop decoder system and the demonstration of its robustness and generalization hold promise for SNN implementations on an ultra-low power neuromorphic chip using the NEF.

  View details for DOI 10.1088/1741-2560/10/3/036008

  View details for PubMedID 23574919

• Design and validation of a real-time spiking-neural-network decoder for brain-machine interfaces JOURNAL OF NEURAL ENGINEERING Dethier, J., Nuyujukian, P., Ryu, S. I., Shenoy, K. V., Boahen, K. 2013; 10 (3)

  View details for DOI 10.1088/1741-2560/10/3/036008

  View details for Web of Science ID 000319510800011

• Neural population dynamics during reaching NATURE Churchland, M. M., Cunningham, J. P., Kaufman, M. T., Foster, J. D., Nuyujukian, P., Ryu, S. I., Shenoy, K. V. 2012; 487 (7405): 51-?

  Abstract

  Most theories of motor cortex have assumed that neural activity represents movement parameters. This view derives from what is known about primary visual cortex, where neural activity represents patterns of light. Yet it is unclear how well the analogy between motor and visual cortex holds. Single-neuron responses in motor cortex are complex, and there is marked disagreement regarding which movement parameters are represented. A better analogy might be with other motor systems, where a common principle is rhythmic neural activity. Here we find that motor cortex responses during reaching contain a brief but strong oscillatory component, something quite unexpected for a non-periodic behaviour. Oscillation amplitude and phase followed naturally from the preparatory state, suggesting a mechanistic role for preparatory neural activity. These results demonstrate an unexpected yet surprisingly simple structure in the population response. This underlying structure explains many of the confusing features of individual neural responses.

  View details for DOI 10.1038/nature11129

  View details for Web of Science ID 000305982900048

  View details for PubMedID 22722855

• A recurrent neural network for closed-loop intracortical brain-machine interface decoders JOURNAL OF NEURAL ENGINEERING Sussillo, D., Nuyujukian, P., Fan, J. M., Kao, J. C., Stavisky, S. D., Ryu, S., Shenoy, K. 2012; 9 (2)

  Abstract

  Recurrent neural networks (RNNs) are useful tools for learning nonlinear relationships in time series data with complex temporal dependences. In this paper, we explore the ability of a simplified type of RNN, one with limited modifications to the internal weights called an echostate network (ESN), to effectively and continuously decode monkey reaches during a standard center-out reach task using a cortical brain-machine interface (BMI) in a closed loop. We demonstrate that the RNN, an ESN implementation termed a FORCE decoder (from first order reduced and controlled error learning), learns the task quickly and significantly outperforms the current state-of-the-art method, the velocity Kalman filter (VKF), using the measure of target acquire time. We also demonstrate that the FORCE decoder generalizes to a more difficult task by successfully operating the BMI in a randomized point-to-point task. The FORCE decoder is also robust as measured by the success rate over extended sessions. Finally, we show that decoded cursor dynamics are more like naturalistic hand movements than those of the VKF. Taken together, these results suggest that RNNs in general, and the FORCE decoder in particular, are powerful tools for BMI decoder applications.

  View details for DOI 10.1088/1741-2560/9/2/026027

  View details for Web of Science ID 000302144100027

  View details for PubMedID 22427488

• HermesE: A 96-Channel Full Data Rate Direct Neural Interface in 0.13 mu m CMOS IEEE JOURNAL OF SOLID-STATE CIRCUITS Gao, H., Walker, R. M., Nuyujukian, P., Makinwa, K. A., Shenoy, K. V., Murmann, B., Meng, T. H. 2012; 47 (4): 1043-1055

  View details for DOI 10.1109/JSSC.2012.2185338

  View details for Web of Science ID 000302494700023

• A framework for relating neural activity to freely moving behavior 2012 ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY (EMBC) Foster, J. D., Nuyujukian, P., Freifeld, O., Ryu, S. I., Black, M. J., Shenoy, K. V. 2012: 2736-2739

  Abstract

  Two research communities, motor systems neuroscience and motor prosthetics, examine the relationship between neural activity in the motor cortex and movement. The former community aims to understand how the brain controls and generates movement; the latter community focuses on how to decode neural activity as control signals for a prosthetic cursor or limb. Both have made progress toward understanding the relationship between neural activity in the motor cortex and behavior. However, these findings are tested using animal models in an environment that constrains behavior to simple, limited movements. These experiments show that, in constrained settings, simple reaching motions can be decoded from small populations of spiking neurons. It is unclear whether these findings hold for more complex, full-body behaviors in unconstrained settings. Here we present the results of freely-moving behavioral experiments from a monkey with simultaneous intracortical recording. We investigated neural firing rates while the monkey performed various tasks such as walking on a treadmill, reaching for food, and sitting idly. We show that even in such an unconstrained and varied context, neural firing rates are well tuned to behavior, supporting findings of basic neuroscience. Further, we demonstrate that the various behavioral tasks can be reliably classified with over 95% accuracy, illustrating the viability of decoding techniques despite significant variation and environmental distractions associated with unconstrained behavior. Such encouraging results hint at potential utility of the freely-moving experimental paradigm.

  View details for Web of Science ID 000313296502238

  View details for PubMedID 23366491

• A brain machine interface control algorithm designed from a feedback control perspective 2012 ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY (EMBC) Gilja, V., Nuyujukian, P., Chestek, C. A., Cunningham, J. P., Yu, B. M., Fan, J. M., Ryu, S. I., Shenoy, K. V. 2012: 1318-1322

  Abstract

  We present a novel brain machine interface (BMI) control algorithm, the recalibrated feedback intention-trained Kalman filter (ReFIT-KF). The design of ReFIT-KF is motivated from a feedback control perspective applied to existing BMI control algorithms. The result is two design innovations that alter the modeling assumptions made by these algorithms and the methods by which these algorithms are trained. In online neural control experiments recording from a 96-electrode array implanted in M1 of a macaque monkey, the ReFIT-KF control algorithm demonstrates large performance improvements over the current state of the art velocity Kalman filter, reducing target acquisition time by a factor of two, while maintaining a 500 ms hold period, thereby increasing the clinical viability of BMI systems.

  View details for Web of Science ID 000313296501144

  View details for PubMedID 23366141

• Long-term stability of neural prosthetic control signals from silicon cortical arrays in rhesus macaque motor cortex Chestek, C. A., Gilja, V., Nuyujukian, P., Foster, J. D., Fan, J. M., Kaufman, M. T., Churchland, M. M., Rivera-Alvidrez, Z., Cunningham, J. P., Ryu, S. I., Shenoy, K. V. IOP PUBLISHING LTD. 2011

  Abstract

  Cortically-controlled prosthetic systems aim to help disabled patients by translating neural signals from the brain into control signals for guiding prosthetic devices. Recent reports have demonstrated reasonably high levels of performance and control of computer cursors and prosthetic limbs, but to achieve true clinical viability, the long-term operation of these systems must be better understood. In particular, the quality and stability of the electrically-recorded neural signals require further characterization. Here, we quantify action potential changes and offline neural decoder performance over 382 days of recording from four intracortical arrays in three animals. Action potential amplitude decreased by 2.4% per month on average over the course of 9.4, 10.4, and 31.7 months in three animals. During most time periods, decoder performance was not well correlated with action potential amplitude (p > 0.05 for three of four arrays). In two arrays from one animal, action potential amplitude declined by an average of 37% over the first 2 months after implant. However, when using simple threshold-crossing events rather than well-isolated action potentials, no corresponding performance loss was observed during this time using an offline decoder. One of these arrays was effectively used for online prosthetic experiments over the following year. Substantial short-term variations in waveforms were quantified using a wireless system for contiguous recording in one animal, and compared within and between days for all three animals. Overall, this study suggests that action potential amplitude declines more slowly than previously supposed, and performance can be maintained over the course of multiple years when decoding from threshold-crossing events rather than isolated action potentials. This suggests that neural prosthetic systems may provide high performance over multiple years in human clinical trials.

  View details for DOI 10.1088/1741-2560/8/4/045005

  View details for Web of Science ID 000292962800009

  View details for PubMedID 21775782

• A closed-loop human simulator for investigating the role of feedback control in brain-machine interfaces JOURNAL OF NEUROPHYSIOLOGY Cunningham, J. P., Nuyujukian, P., Gilja, V., Chestek, C. A., Ryu, S. I., Shenoy, K. V. 2011; 105 (4): 1932-1949

  Abstract

  Neural prosthetic systems seek to improve the lives of severely disabled people by decoding neural activity into useful behavioral commands. These systems and their decoding algorithms are typically developed "offline," using neural activity previously gathered from a healthy animal, and the decoded movement is then compared with the true movement that accompanied the recorded neural activity. However, this offline design and testing may neglect important features of a real prosthesis, most notably the critical role of feedback control, which enables the user to adjust neural activity while using the prosthesis. We hypothesize that understanding and optimally designing high-performance decoders require an experimental platform where humans are in closed-loop with the various candidate decode systems and algorithms. It remains unexplored the extent to which the subject can, for a particular decode system, algorithm, or parameter, engage feedback and other strategies to improve decode performance. Closed-loop testing may suggest different choices than offline analyses. Here we ask if a healthy human subject, using a closed-loop neural prosthesis driven by synthetic neural activity, can inform system design. We use this online prosthesis simulator (OPS) to optimize "online" decode performance based on a key parameter of a current state-of-the-art decode algorithm, the bin width of a Kalman filter. First, we show that offline and online analyses indeed suggest different parameter choices. Previous literature and our offline analyses agree that neural activity should be analyzed in bins of 100- to 300-ms width. OPS analysis, which incorporates feedback control, suggests that much shorter bin widths (25-50 ms) yield higher decode performance. Second, we confirm this surprising finding using a closed-loop rhesus monkey prosthetic system. These findings illustrate the type of discovery made possible by the OPS, and so we hypothesize that this novel testing approach will help in the design of prosthetic systems that will translate well to human patients.

  View details for DOI 10.1152/jn.00503.2010

  View details for Web of Science ID 000289620500044

  View details for PubMedID 20943945

• Monkey Models for Brain-Machine Interfaces: The Need for Maintaining Diversity Nuyujukian, P., Fan, J. M., Gilja, V., Kalanithi, P. S., Chestek, C. A., Shenoy, K. V. IEEE. 2011: 1301-1305

  Abstract

  Brain-machine interfaces (BMIs) aim to help disabled patients by translating neural signals from the brain into control signals for guiding prosthetic arms, computer cursors, and other assistive devices. Animal models are central to the development of these systems and have helped enable the successful translation of the first generation of BMIs. As we move toward next-generation systems, we face the question of which animal models will aid broader patient populations and achieve even higher performance, robustness, and functionality. We review here four general types of rhesus monkey models employed in BMI research, and describe two additional, complementary models. Given the physiological diversity of neurological injury and disease, we suggest a need to maintain the current diversity of animal models and to explore additional alternatives, as each mimic different aspects of injury or disease.

  View details for Web of Science ID 000298810001110

  View details for PubMedID 22254555

• Spiking Neural Network Decoder for Brain-Machine Interfaces 2011 5TH INTERNATIONAL IEEE/EMBS CONFERENCE ON NEURAL ENGINEERING (NER) Dethier, J., Gilja, V., Nuyujukian, P., Elassaad, S. A., Shenoy, K. V., Boahen, K. 2011: 396-399

  View details for Web of Science ID 000298735800094

• Combining Wireless Neural Recording and Video Capture for the Analysis of Natural Gait 2011 5TH INTERNATIONAL IEEE/EMBS CONFERENCE ON NEURAL ENGINEERING (NER) Foster, J. D., Freifeld, O., Nuyujukian, P., Ryu, S. I., Black, M. J., Shenoy, K. V. 2011: 613-616

  View details for Web of Science ID 000298735800147

• Autonomous head-mounted electrophysiology systems for freely behaving primates CURRENT OPINION IN NEUROBIOLOGY Gilja, V., Chestek, C. A., Nuyujukian, P., Foster, J., Shenoy, K. V. 2010; 20 (5): 676-686

  Abstract

  Recent technological advances have led to new light-weight battery-operated systems for electrophysiology. Such systems are head mounted, run for days without experimenter intervention, and can record and stimulate from single or multiple electrodes implanted in a freely behaving primate. Here we discuss existing systems, studies that use them, and how they can augment traditional, physically restrained, 'in-rig' electrophysiology. With existing technical capabilities, these systems can acquire multiple signal classes, such as spikes, local field potential, and electromyography signals, and can stimulate based on real-time processing of recorded signals. Moving forward, this class of technologies, along with advances in neural signal processing and behavioral monitoring, have the potential to dramatically expand the scope and scale of electrophysiological studies.

  View details for DOI 10.1016/j.conb.2010.06.007

  View details for Web of Science ID 000283481100022

  View details for PubMedID 20655733

• Embedded Neural Recording With TinyOS-Based Wireless-Enabled Processor Modules IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING Farshchi, S., Pesterev, A., Nuyujukian, P., Guenterberg, E., Mody, I., Judy, J. W. 2010; 18 (2): 134-141

  Abstract

  To create a wireless neural recording system that can benefit from the continuous advancements being made in embedded microcontroller and communications technologies, an embedded-system-based architecture for wireless neural recording has been designed, fabricated, and tested. The system consists of commercial-off-the-shelf wireless-enabled processor modules (motes) for communicating the neural signals, and a back-end database server and client application for archiving and browsing the neural signals. A neural-signal-acquisition application has been developed to enable the mote to either acquire neural signals at a rate of 4000 12-bit samples per second, or detect and transmit spike heights and widths sampled at a rate of 16670 12-bit samples per second on a single channel. The motes acquire neural signals via a custom low-noise neural-signal amplifier with adjustable gain and high-pass corner frequency that has been designed, and fabricated in a 1.5-microm CMOS process. In addition to browsing acquired neural data, the client application enables the user to remotely toggle modes of operation (real-time or spike-only), as well as amplifier gain and high-pass corner frequency.

  View details for DOI 10.1109/TNSRE.2009.2039606

  View details for Web of Science ID 000277184700005

  View details for PubMedID 20071270

• Wireless Neural Recording With Single Low-Power Integrated Circuit IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING Harrison, R. R., Kier, R. J., Chestek, C. A., Gilja, V., Nuyujukian, P., Ryu, S., Greger, B., Solzbacher, F., Shenoy, K. V. 2009; 17 (4): 322-329

  Abstract

  We present benchtop and in vivo experimental results from an integrated circuit designed for wireless implantable neural recording applications. The chip, which was fabricated in a commercially available 0.6- mum 2P3M BiCMOS process, contains 100 amplifiers, a 10-bit analog-to-digital converter (ADC), 100 threshold-based spike detectors, and a 902-928 MHz frequency-shift-keying (FSK) transmitter. Neural signals from a selected amplifier are sampled by the ADC at 15.7 kSps and telemetered over the FSK wireless data link. Power, clock, and command signals are sent to the chip wirelessly over a 2.765-MHz inductive (coil-to-coil) link. The chip is capable of operating with only two off-chip components: a power/command receiving coil and a 100-nF capacitor.

  View details for DOI 10.1109/TNSRE.2009.2023298

  View details for Web of Science ID 000268900300003

  View details for PubMedID 19497825

• HermesC: Low-Power Wireless Neural Recording System for Freely Moving Primates IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING Chestek, C. A., Gilja, V., Nuyujukian, P., Kier, R. J., Solzbacher, F., Ryu, S. I., Harrison, R. R., Shenoy, K. V. 2009; 17 (4): 330-338

  Abstract

  Neural prosthetic systems have the potential to restore lost functionality to amputees or patients suffering from neurological injury or disease. Current systems have primarily been designed for immobile patients, such as tetraplegics functioning in a rather static, carefully tailored environment. However, an active patient such as amputee in a normal dynamic, everyday environment may be quite different in terms of the neural control of movement. In order to study motor control in a more unconstrained natural setting, we seek to develop an animal model of freely moving humans. Therefore, we have developed and tested HermesC-INI3, a system for recording and wirelessly transmitting neural data from electrode arrays implanted in rhesus macaques who are freely moving. This system is based on the integrated neural interface (INI3) microchip which amplifies, digitizes, and transmits neural data across a approximately 900 MHz wireless channel. The wireless transmission has a range of approximately 4 m in free space. All together this device consumes 15.8 mA and 63.2 mW. On a single 2 A-hr battery pack, this device runs contiguously for approximately six days. The smaller size and power consumption of the custom IC allows for a smaller package (51 x 38 x 38 mm (3)) than previous primate systems. The HermesC-INI3 system was used to record and telemeter one channel of broadband neural data at 15.7 kSps from a monkey performing routine daily activities in the home cage.

  View details for DOI 10.1109/TNSRE.2009.2023293

  View details for Web of Science ID 000268900300004

  View details for PubMedID 19497829

• Neural Prosthetic Systems: Current Problems and Future Directions Chestek, C. A., Cunningham, J. P., Gilja, V., Nuyujukian, P., Ryu, S. I., Shenoy, K. V. IEEE. 2009: 3369-3375

  Abstract

  By decoding neural activity into useful behavioral commands, neural prosthetic systems seek to improve the lives of severely disabled human patients. Motor decoding algorithms, which map neural spiking data to control parameters of a device such as a prosthetic arm, have received particular attention in the literature. Here, we highlight several outstanding problems that exist in most current approaches to decode algorithm design. These include two problems that we argue will unlikely result in further dramatic increases in performance, specifically spike sorting and spiking models. We also discuss three issues that have been less examined in the literature, and we argue that addressing these issues may result in dramatic future increases in performance. These include: non-stationarity of recorded waveforms, limitations of a linear mappings between neural activity and movement kinematics, and the low signal to noise ratio of the neural data. We demonstrate these problems with data from 39 experimental sessions with a non-human primate performing reaches and with recent literature. In all, this study suggests that research in cortically-controlled prosthetic systems may require reprioritization to achieve performance that is acceptable for a clinically viable human system.

  View details for Web of Science ID 000280543602209

  View details for PubMedID 19963796

• HermesC: RF wireless low-power neural recording system for freely behaving primates PROCEEDINGS OF 2008 IEEE INTERNATIONAL SYMPOSIUM ON CIRCUITS AND SYSTEMS, VOLS 1-10 Chestek, C. A., Gija, V., Nuyujukian, P., Ryu, S. I., Shenoy, K. V., Kier, R. J., Solzbacher, F., Harrison, R. R. 2008: 1752-1755

  View details for Web of Science ID 000258532101169

• Wireless neural signal acquisition with single low-power integrated circuit PROCEEDINGS OF 2008 IEEE INTERNATIONAL SYMPOSIUM ON CIRCUITS AND SYSTEMS, VOLS 1-10 Harrison, R. R., Kier, R. J., Greger, B., Solzbacher, F., Chestek, C. A., Gija, V., Nuyujukian, P., Ryu, S. I., Shenoy, K. V. 2008: 1748-1751

  View details for Web of Science ID 000258532101168

• Bi-Fi: An embedded sensor/system architecture for remote biological monitoring IEEE TRANSACTIONS ON INFORMATION TECHNOLOGY IN BIOMEDICINE Farshchi, S., Pesterev, A., Nuyujukian, P. H., Mody, I., Judy, J. W. 2007; 11 (6): 611-618

  Abstract

  Wireless-enabled processor modules intended for communicating low-frequency phenomena (i.e., temperature, humidity, and ambient light) have been enabled to acquire and transmit multiple biological signals in real time, which has been achieved by using computationally efficient data acquisition, filtering, and compression algorithms, and interfacing the modules with biological interface hardware. The sensor modules can acquire and transmit raw biological signals at a rate of 32 kb/s, which is near the hardware limit of the modules. Furthermore, onboard signal processing enables one channel, sampled at a rate of 4000 samples/s at 12-bit resolution, to be compressed via adaptive differential-pulse-code modulation (ADPCM) and transmitted in real time. In addition, the sensors can be configured to filter and transmit individual time-referenced "spike" waveforms, or to transmit the spike height and width for alleviating network traffic and increasing battery life. The system is capable of acquiring eight channels of analog signals as well as data via an asynchronous serial connection. A back-end server archives the biological data received via networked gateway sensors, and hosts them to a client application that enables users to browse recorded data. The system also acquires, filters, and transmits oxygen saturation and pulse rate via a commercial-off-the-shelf interface board. The system architecture can be configured for performing real-time nonobtrusive biological monitoring of humans or rodents. This paper demonstrates that low-power, computational, and bandwidth-constrained wireless-enabled platforms can indeed be leveraged for wireless biosignal monitoring.

  View details for DOI 10.1109/TITB.2007.897600

  View details for Web of Science ID 000250929100002

  View details for PubMedID 18046936

• A TinyOS-enabled MICA2-based wireless neural interface IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING Farshchi, S., Nuyujukian, P. H., Pesterev, A., Mody, I., Judy, J. W. 2006; 53 (7): 1416-1424

  Abstract

  Existing approaches used to develop compact low-power multichannel wireless neural recording systems range from creating custom-integrated circuits to assembling commercial-off-the-shelf (COTS) PC-based components. Custom-integrated-circuit designs yield extremely compact and low-power devices at the expense of high development and upgrade costs and turn-around times, while assembling COTS-PC-technology yields high performance at the expense of large system size and increased power consumption. To achieve a balance between implementing an ultra-compact custom-fabricated neural transceiver and assembling COTS-PC-technology, an overlay of a neural interface upon the TinyOS-based MICA2 platform is described. The system amplifies, digitally encodes, and transmits neural signals real-time at a rate of 9.6 kbps, while consuming less than 66 mW of power. The neural signals are received and forwarded to a client PC over a serial connection. This data rate can be divided for recording on up to 6 channels, with a resolution of 8 bits/sample. This work demonstrates the strengths and limitations of the TinyOS-based sensor technology as a foundation for chronic remote biological monitoring applications and, thus, provides an opportunity to create a system that can leverage from the frequent networking and communications advancements being made by the global TinyOS-development community.

  View details for DOI 10.1109/TMBE.2006.873760

  View details for Web of Science ID 000238711600021

  View details for PubMedID 16830946

• A TinyOS-based wireless neural sensing, archiving, and hosting system 2005 2ND INTERNATINOAL IEEE/EMBS CONFERENCE ON NEURAL ENGINEERING Farshchi, S., Nuyujukian, P. H., Pesterev, A., Mody, I., Judy, J. W. 2005: 671-674

  View details for Web of Science ID 000229610400179