The design of our artificial retina system is based on knowledge acquired in our unique laboratory setting. The Chichilnisky lab has spent many years studying how light stimuli are encoded by neural circuitry in the healthy retina, specifically, the non-human primate retina, which resembles the human retina much more closely than other animal models. Our goal is to use the information we and others have gathered to build an advanced retinal implant that can produce a naturalistic visual signal. Our main focus is on ganglion cells, the output neurons of the retina, which largely survive the retinal degeneration process. Ganglion cells generate visual representations of light inputs and transmit these encoded representations towards specific target areas in the brain. The human retina contains around 1 million ganglion cells, and all visual experiences ultimately arise from the signals transmitted by these cells. By first learning how patterns of activity in many ganglion cells represent a visual scene, we can mimic the neural retinal code using our artificial implant. If we accurately reproduce the retinal code, the brain can accurately perceive the visual image.
Ganglion cells come in many types
Importantly, ganglion cells are not a single cell population. The human retina contains about 20 distinct types of ganglion cells and each type responds differently to light stimuli. For example, some cells respond to the presence of light, while other cells respond to the absence of light. Some cells convey color information, while others are more sensitive to flickering light. Some are characterized by small and smooth receptive fields while others show large and irregular receptive fields. Some show very fast responses to light, others are much slower to respond. Some types are very numerous, others are much less abundant. Recognizing this heterogeneity forms the basis of our artificial retina and sets our approach apart from all other approaches.
Each type of ganglion cell forms a uniform lattice, or mosaic, across the retina, thereby sampling the entire visual scene, and carries a specific aspect of the visual stimulus to its specific targets the brain. The different ganglion cell types have been likened to the various instruments in an orchestra, each contributing uniquely to the musical score. A faithful reproduction of music is only possible by having each instrument play its part with correct timing and order. Similarly, to reproduce a meaningful neural retinal code, it is crucial to respect the specificity and selectivity of the diverse retinal ganglion cell types.
Key questions we are asking
We use large-scale multi-electrode recording from the non-human primate retina to characterize normal light-evoked activity in hundreds of retinal ganglion cells of multiple types simultaneously. We then evoke similar patterns of activity by fine-grained, patterned electrical stimulation. Finally, we evaluate how effectively we have mimicked the neural code by using machine learning methods to estimate the visual image that would be perceived by the subject in response to this electrical stimulation. This approach constitutes our laboratory prototype for an artificial retina.
We focus on several questions:
- What distinct aspects of vision are encoded by the diverse types of ganglion cells?
- How can computational models accurately mimic these retinal signals?
- How can we precisely target individual ganglion cells using an implant?
- How can the distinct cell types be recognized and separately targeted by the implant?
- How can we engineer an implant that faithfully reproduces normal vision in blind patients?
We anticipate that in addition to restoring vision, the artificial retina will allow us to transmit visual information to the brain in ways that are not possible with light stimulation. This will open the door to visual augmentation – creating visual sensations that were never before possible. Our understanding of the retinal circuitry and how to interface to it effectively will be relevant for developing other interfaces to the brain – both for treating disease and for augmenting human capabilities.
Explore our approach on a more in-depth level by selecting the Details tab on this page.
Replacing visual signalling with electrical stimulation
To restore vision, we stimulate retinal ganglion cells directly with minute amounts of electrical current. An array of hundreds to thousands of electrodes delivers precisely timed electrical stimuli to the retina. Our research over the past 15 years has shown that the use of small electrodes (typically less than 10 μm in diameter) produces very selective activation of individual ganglion cells. We are also exploring the use of curved fine microwire arrays that slightly penetrate the retinal surface and position the stimulating electrodes in even closer proximity to ganglion cell bodies.
Cell-type specific stimulation and building a dictionary
Instead of indiscriminately activating any ganglion cells near a stimulating electrode, our approach respects the diversity of ganglion cell types. Over the past several years, we have used our laboratory prototype artificial retina to demonstrate that we can identify and target all the major retinal ganglion cell types at single-cell and single-spike resolution. We propose a two-step process to achieve high-acuity artificial vision: an initial calibration step, followed by run-time activation of the implant.
Calibration: We use the implanted electrode array to record and identify the many distinct ganglion cell types. We then assemble a map of which electrodes can efficiently stimulate particular ganglion cells under the array. This step is nearly fully automated and takes advantage of our implant's ability to both stimulate cells and simultaneously record their responses. We call the resulting calibration map the dictionary of achievable electrical stimulation patterns and this dictionary is stored in the device after the calibration step has completed.
Run-time: To reproduce a desired artificial visual stimulus, the implant uses the dictionary of achievable electrical stimulation patterns from the on-device memory and instructs the stimulation array to activate the appropriate ganglion cells at the appropriate times, effectively transforming any incoming visual scene into exquisitely timed activity in the output of the retina.
Estimating ganglion cell properties
In our design process, we rely heavily on computer models to predict how our cell-type specific stimulation approach affects perception. Inferring a patient's perception using computer simulations allows us to fine-tune all aspects of the artificial retina – such as the size and spacing of electrodes, the number of targeted cell types, and the timing of electrical pulses. It also allows us to incorporate what is known about the disease state into these design considerations.
Putting it all together: our concept device
Our proposed device includes several components, some of which are mounted on specially designed glasses, while others reside on or inside the eye. All connectivity is accomplished wirelessly using either radio frequency (RF) or ultrasound transmission. The advantage of this design lies in the minimalist nature of the permanent implant, while the external components are easily accessible and upgradeable. Changes to the software algorithms can be simply uploaded to the device without altering the implanted microarray, and improvements in most of the hardware can be accomplished with new glasses. A powerful signal processor receives data from several sensors and from the implanted electrode array, which can record retinal signals. After calibration, it transforms visual camera signals into stimulation pulse commands sent wirelessly to the implant.
Restoring vision to people with incurable blindness can profoundly alter the lives of millions, restoring their ability to function in society as they are accustomed to and eliminating one of the greatest challenges of aging.
By replacing light-evoked with electrical stimulation we aim to provide a treatment for several retinal degenerative diseases. The main targets are age-related macular degeneration (AMD) and retinitis pigmentosa (RP). In the US alone, AMD affects 2 million people and RP affects 75,000 people.
In AMD, the high-acuity center of vision (macula) is impacted by photoreceptor degeneration, leading to the appearance of large yellow or white spots in this area. In contrast, RP is characterized by peripheral retinal degeneration and presents as dark pigment deposits that spare the center of vision, but eventually leads to complete loss of vision.
Surgical implant placement
The human fovea contains the highest density of photoreceptors and ganglion cells and may at first glance appear to be the implant location of choice. Indeed, our ultimate goal is to develop a foveal implant, but initial devices will be placed slightly peripheral to the fovea. This is because the foveal ganglion cell layer is composed of five or more stacked rows of cells, making targeted stimulation of a high fraction of ganglion cells at single-cell resolution more difficult. The target location for our first implantations will be the retinal raphe, a near-foveal area of the retina central enough to provide high-resolution artificial vision while avoiding the complexity of the fovea. However, we are actively developing methods aimed at moving toward the fovea.
Our approach has the potential to spawn a new generation of electronic implants to restore neuronal function in disorders such as paralysis, memory loss, psychiatric disorders, and more. This is because, what we learn about how to interface effectively to neural circuitry at a cellular resolution will be applicable for developing other interfaces to the brain: the basic architecture of the retina and the brain are similar with multiple distinct cell types performing different functions and sending their output to different targets. The retina is the right place to start because we know more about its function than almost any other part of the nervous system.
Another potential impact of our technology is augmentation of visual function. In addition to restoring normal vision, a high-resolution retinal implant could convey information from invisible portions of the light spectrum, text or images, and any other source of visual signals. It might also enable new forms of multitasking by relaying different information to distinct ganglion cell types simultaneously. The development of this technology will open doors to exploration of how we can most effectively harness the visual pathways, which provide a very high bandwidth interface to the brain.
Successful implementation of our device will furthermore provide a number of clinical and basic science lessons that heretofore remained inaccessible. For example, we will be able to decipher what distinct aspects of vision are conveyed separately to the brain by different cell types, providing a unique and powerful research instrument for understanding the diverse retinal pathways and how they contribute to vision.