Paul Nuyujukian

Abstract

Objective. Cortically-controlled motor prostheses aim to restore functions lost to neurological disease and injury. Several proof of concept demonstrations have shown encouraging results, but barriers to clinical translation still remain. In particular, intracortical prostheses must satisfy stringent power dissipation constraints so as not to damage cortex. Approach. One possible solution is to use ultra-low power neuromorphic chips to decode neural signals for these intracortical implants. The first step is to explore in simulation the feasibility of translating decoding algorithms for brain-machine interface (BMI) applications into spiking neural networks (SNNs). Main results. Here we demonstrate the validity of the approach by implementing an existing Kalman-filter-based decoder in a simulated SNN using the Neural Engineering Framework (NEF), a general method for mapping control algorithms onto SNNs. To measure this system's robustness and generalization, we tested it online in closed-loop BMI experiments with two rhesus monkeys. Across both monkeys, a Kalman filter implemented using a 2000-neuron SNN has comparable performance to that of a Kalman filter implemented using standard floating point techniques. Significance. These results demonstrate the tractability of SNN implementations of statistical signal processing algorithms on different monkeys and for several tasks, suggesting that a SNN decoder, implemented on a neuromorphic chip, may be a feasible computational platform for low-power fully-implanted prostheses. The validation of this closed-loop decoder system and the demonstration of its robustness and generalization hold promise for SNN implementations on an ultra-low power neuromorphic chip using the NEF.

View details for DOI 10.1088/1741-2560/10/3/036008

View details for PubMedID 23574919

Abstract

Most theories of motor cortex have assumed that neural activity represents movement parameters. This view derives from what is known about primary visual cortex, where neural activity represents patterns of light. Yet it is unclear how well the analogy between motor and visual cortex holds. Single-neuron responses in motor cortex are complex, and there is marked disagreement regarding which movement parameters are represented. A better analogy might be with other motor systems, where a common principle is rhythmic neural activity. Here we find that motor cortex responses during reaching contain a brief but strong oscillatory component, something quite unexpected for a non-periodic behaviour. Oscillation amplitude and phase followed naturally from the preparatory state, suggesting a mechanistic role for preparatory neural activity. These results demonstrate an unexpected yet surprisingly simple structure in the population response. This underlying structure explains many of the confusing features of individual neural responses.

View details for DOI 10.1038/nature11129

View details for Web of Science ID 000305982900048

View details for PubMedID 22722855

Abstract

Recurrent neural networks (RNNs) are useful tools for learning nonlinear relationships in time series data with complex temporal dependences. In this paper, we explore the ability of a simplified type of RNN, one with limited modifications to the internal weights called an echostate network (ESN), to effectively and continuously decode monkey reaches during a standard center-out reach task using a cortical brain-machine interface (BMI) in a closed loop. We demonstrate that the RNN, an ESN implementation termed a FORCE decoder (from first order reduced and controlled error learning), learns the task quickly and significantly outperforms the current state-of-the-art method, the velocity Kalman filter (VKF), using the measure of target acquire time. We also demonstrate that the FORCE decoder generalizes to a more difficult task by successfully operating the BMI in a randomized point-to-point task. The FORCE decoder is also robust as measured by the success rate over extended sessions. Finally, we show that decoded cursor dynamics are more like naturalistic hand movements than those of the VKF. Taken together, these results suggest that RNNs in general, and the FORCE decoder in particular, are powerful tools for BMI decoder applications.

View details for DOI 10.1088/1741-2560/9/2/026027

View details for Web of Science ID 000302144100027

View details for PubMedID 22427488

View details for DOI 10.1109/JSSC.2012.2185338

View details for Web of Science ID 000302494700023

Abstract

Two research communities, motor systems neuroscience and motor prosthetics, examine the relationship between neural activity in the motor cortex and movement. The former community aims to understand how the brain controls and generates movement; the latter community focuses on how to decode neural activity as control signals for a prosthetic cursor or limb. Both have made progress toward understanding the relationship between neural activity in the motor cortex and behavior. However, these findings are tested using animal models in an environment that constrains behavior to simple, limited movements. These experiments show that, in constrained settings, simple reaching motions can be decoded from small populations of spiking neurons. It is unclear whether these findings hold for more complex, full-body behaviors in unconstrained settings. Here we present the results of freely-moving behavioral experiments from a monkey with simultaneous intracortical recording. We investigated neural firing rates while the monkey performed various tasks such as walking on a treadmill, reaching for food, and sitting idly. We show that even in such an unconstrained and varied context, neural firing rates are well tuned to behavior, supporting findings of basic neuroscience. Further, we demonstrate that the various behavioral tasks can be reliably classified with over 95% accuracy, illustrating the viability of decoding techniques despite significant variation and environmental distractions associated with unconstrained behavior. Such encouraging results hint at potential utility of the freely-moving experimental paradigm.

View details for Web of Science ID 000313296502238

View details for PubMedID 23366491

Abstract

We present a novel brain machine interface (BMI) control algorithm, the recalibrated feedback intention-trained Kalman filter (ReFIT-KF). The design of ReFIT-KF is motivated from a feedback control perspective applied to existing BMI control algorithms. The result is two design innovations that alter the modeling assumptions made by these algorithms and the methods by which these algorithms are trained. In online neural control experiments recording from a 96-electrode array implanted in M1 of a macaque monkey, the ReFIT-KF control algorithm demonstrates large performance improvements over the current state of the art velocity Kalman filter, reducing target acquisition time by a factor of two, while maintaining a 500 ms hold period, thereby increasing the clinical viability of BMI systems.

View details for Web of Science ID 000313296501144

View details for PubMedID 23366141

Abstract

Neural prosthetic systems seek to improve the lives of severely disabled people by decoding neural activity into useful behavioral commands. These systems and their decoding algorithms are typically developed "offline," using neural activity previously gathered from a healthy animal, and the decoded movement is then compared with the true movement that accompanied the recorded neural activity. However, this offline design and testing may neglect important features of a real prosthesis, most notably the critical role of feedback control, which enables the user to adjust neural activity while using the prosthesis. We hypothesize that understanding and optimally designing high-performance decoders require an experimental platform where humans are in closed-loop with the various candidate decode systems and algorithms. It remains unexplored the extent to which the subject can, for a particular decode system, algorithm, or parameter, engage feedback and other strategies to improve decode performance. Closed-loop testing may suggest different choices than offline analyses. Here we ask if a healthy human subject, using a closed-loop neural prosthesis driven by synthetic neural activity, can inform system design. We use this online prosthesis simulator (OPS) to optimize "online" decode performance based on a key parameter of a current state-of-the-art decode algorithm, the bin width of a Kalman filter. First, we show that offline and online analyses indeed suggest different parameter choices. Previous literature and our offline analyses agree that neural activity should be analyzed in bins of 100- to 300-ms width. OPS analysis, which incorporates feedback control, suggests that much shorter bin widths (25-50 ms) yield higher decode performance. Second, we confirm this surprising finding using a closed-loop rhesus monkey prosthetic system. These findings illustrate the type of discovery made possible by the OPS, and so we hypothesize that this novel testing approach will help in the design of prosthetic systems that will translate well to human patients.

View details for DOI 10.1152/jn.00503.2010

View details for Web of Science ID 000289620500044

View details for PubMedID 20943945

View details for Web of Science ID 000298735800094

View details for Web of Science ID 000298735800147

Abstract

Recent technological advances have led to new light-weight battery-operated systems for electrophysiology. Such systems are head mounted, run for days without experimenter intervention, and can record and stimulate from single or multiple electrodes implanted in a freely behaving primate. Here we discuss existing systems, studies that use them, and how they can augment traditional, physically restrained, 'in-rig' electrophysiology. With existing technical capabilities, these systems can acquire multiple signal classes, such as spikes, local field potential, and electromyography signals, and can stimulate based on real-time processing of recorded signals. Moving forward, this class of technologies, along with advances in neural signal processing and behavioral monitoring, have the potential to dramatically expand the scope and scale of electrophysiological studies.

View details for DOI 10.1016/j.conb.2010.06.007

View details for Web of Science ID 000283481100022

View details for PubMedID 20655733

Abstract

To create a wireless neural recording system that can benefit from the continuous advancements being made in embedded microcontroller and communications technologies, an embedded-system-based architecture for wireless neural recording has been designed, fabricated, and tested. The system consists of commercial-off-the-shelf wireless-enabled processor modules (motes) for communicating the neural signals, and a back-end database server and client application for archiving and browsing the neural signals. A neural-signal-acquisition application has been developed to enable the mote to either acquire neural signals at a rate of 4000 12-bit samples per second, or detect and transmit spike heights and widths sampled at a rate of 16670 12-bit samples per second on a single channel. The motes acquire neural signals via a custom low-noise neural-signal amplifier with adjustable gain and high-pass corner frequency that has been designed, and fabricated in a 1.5-microm CMOS process. In addition to browsing acquired neural data, the client application enables the user to remotely toggle modes of operation (real-time or spike-only), as well as amplifier gain and high-pass corner frequency.

View details for DOI 10.1109/TNSRE.2009.2039606

View details for Web of Science ID 000277184700005

View details for PubMedID 20071270

Abstract

We present benchtop and in vivo experimental results from an integrated circuit designed for wireless implantable neural recording applications. The chip, which was fabricated in a commercially available 0.6- mum 2P3M BiCMOS process, contains 100 amplifiers, a 10-bit analog-to-digital converter (ADC), 100 threshold-based spike detectors, and a 902-928 MHz frequency-shift-keying (FSK) transmitter. Neural signals from a selected amplifier are sampled by the ADC at 15.7 kSps and telemetered over the FSK wireless data link. Power, clock, and command signals are sent to the chip wirelessly over a 2.765-MHz inductive (coil-to-coil) link. The chip is capable of operating with only two off-chip components: a power/command receiving coil and a 100-nF capacitor.

View details for DOI 10.1109/TNSRE.2009.2023298

View details for Web of Science ID 000268900300003

View details for PubMedID 19497825

Abstract

Neural prosthetic systems have the potential to restore lost functionality to amputees or patients suffering from neurological injury or disease. Current systems have primarily been designed for immobile patients, such as tetraplegics functioning in a rather static, carefully tailored environment. However, an active patient such as amputee in a normal dynamic, everyday environment may be quite different in terms of the neural control of movement. In order to study motor control in a more unconstrained natural setting, we seek to develop an animal model of freely moving humans. Therefore, we have developed and tested HermesC-INI3, a system for recording and wirelessly transmitting neural data from electrode arrays implanted in rhesus macaques who are freely moving. This system is based on the integrated neural interface (INI3) microchip which amplifies, digitizes, and transmits neural data across a approximately 900 MHz wireless channel. The wireless transmission has a range of approximately 4 m in free space. All together this device consumes 15.8 mA and 63.2 mW. On a single 2 A-hr battery pack, this device runs contiguously for approximately six days. The smaller size and power consumption of the custom IC allows for a smaller package (51 x 38 x 38 mm (3)) than previous primate systems. The HermesC-INI3 system was used to record and telemeter one channel of broadband neural data at 15.7 kSps from a monkey performing routine daily activities in the home cage.

View details for DOI 10.1109/TNSRE.2009.2023293

View details for Web of Science ID 000268900300004

View details for PubMedID 19497829

View details for Web of Science ID 000258532101169

View details for Web of Science ID 000258532101168

Abstract

Wireless-enabled processor modules intended for communicating low-frequency phenomena (i.e., temperature, humidity, and ambient light) have been enabled to acquire and transmit multiple biological signals in real time, which has been achieved by using computationally efficient data acquisition, filtering, and compression algorithms, and interfacing the modules with biological interface hardware. The sensor modules can acquire and transmit raw biological signals at a rate of 32 kb/s, which is near the hardware limit of the modules. Furthermore, onboard signal processing enables one channel, sampled at a rate of 4000 samples/s at 12-bit resolution, to be compressed via adaptive differential-pulse-code modulation (ADPCM) and transmitted in real time. In addition, the sensors can be configured to filter and transmit individual time-referenced "spike" waveforms, or to transmit the spike height and width for alleviating network traffic and increasing battery life. The system is capable of acquiring eight channels of analog signals as well as data via an asynchronous serial connection. A back-end server archives the biological data received via networked gateway sensors, and hosts them to a client application that enables users to browse recorded data. The system also acquires, filters, and transmits oxygen saturation and pulse rate via a commercial-off-the-shelf interface board. The system architecture can be configured for performing real-time nonobtrusive biological monitoring of humans or rodents. This paper demonstrates that low-power, computational, and bandwidth-constrained wireless-enabled platforms can indeed be leveraged for wireless biosignal monitoring.

View details for DOI 10.1109/TITB.2007.897600

View details for Web of Science ID 000250929100002

View details for PubMedID 18046936

Abstract

Existing approaches used to develop compact low-power multichannel wireless neural recording systems range from creating custom-integrated circuits to assembling commercial-off-the-shelf (COTS) PC-based components. Custom-integrated-circuit designs yield extremely compact and low-power devices at the expense of high development and upgrade costs and turn-around times, while assembling COTS-PC-technology yields high performance at the expense of large system size and increased power consumption. To achieve a balance between implementing an ultra-compact custom-fabricated neural transceiver and assembling COTS-PC-technology, an overlay of a neural interface upon the TinyOS-based MICA2 platform is described. The system amplifies, digitally encodes, and transmits neural signals real-time at a rate of 9.6 kbps, while consuming less than 66 mW of power. The neural signals are received and forwarded to a client PC over a serial connection. This data rate can be divided for recording on up to 6 channels, with a resolution of 8 bits/sample. This work demonstrates the strengths and limitations of the TinyOS-based sensor technology as a foundation for chronic remote biological monitoring applications and, thus, provides an opportunity to create a system that can leverage from the frequent networking and communications advancements being made by the global TinyOS-development community.

View details for DOI 10.1109/TMBE.2006.873760

View details for Web of Science ID 000238711600021

View details for PubMedID 16830946

View details for Web of Science ID 000229610400179