# US 6,400,989 B1 · Adaptive sensory-motor encoder for visual or acoustic prosthesis > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 6,400,989 B1 - Document ID: US6400989B1 - Status: Granted - Kind: B1 - Application number: 09/367,030 - Filed: 2000-05-30 - Granted: 2002-06-04 - Earliest claimed: 1997-02-21 - Assignees: Intelligent Implants Limited - Inventors: Rolf Eckmiller - Web page: https://patentgrep.com/patent/US6400989B1 ## CPC - A61N 1/36046: {of the eye} - A61N 1/36038: {Cochlear stimulation} - A61N 1/0543: {Retinal electrodes} - A61N: ELECTROTHERAPY; MAGNETOTHERAPY; RADIATION THERAPY; ULTRASOUND THERAPY (measurement of bioelectric currents A61B; surgical instruments, devices or methods for transferring non-mechanical forms of energy to or from the body A61B18/00; anaesthetic apparatus in general A61M; incandescent lamps H01K; infrared radiators for heating H05B) ## Abstract The invention describes an adaptive, sensory-motor encoder for a visual prosthesis or for an acoustic prosthesis and equipped with a central control unit for signal processing functions, monitoring functions, control functions and external intervention functions as well as with a group of adaptive spatio-temporal filters for the conversion of sensor signals into stimulation impulse sequences, whereby a bi-directional interface is provided for coupling the encoder with an implantable microstructure (2) for stimulation of nerve or glial tissue on the one hand, and on the other hand for function monitoring of brain function. ## Claims ### Claim 1 A visual or acoustic prosthesis comprising: an adaptive sensory-motor encoder, comprising: a central control unit for performing signal processing functions, monitoring functions, control functions, and external pick-up functions, the central control unit including a group of adaptive spatio-temporal filters for converting sensory signals into stimulation pulse sequences; an implantable microstructure for providing stimulation to nerve or glial tissue and for functional monitoring of neural functions; and a bi-directional interface coupling the encoder to the microstructure, through which at least one of stimulation and control signals are provided to the microstructure and monitoring signals are provided to the encoder. ### Claim 2 (depends on 1) The prosthesis according to claim 1, wherein the control unit includes in a perception-based dialog process receptive field (RF) filters, said RF filters comprising adjustable spatio-temporal filters with receptive field properties. ### Claim 3 (depends on 2) The prosthesis according to claim 2, wherein the RF filters are associated with time delay elements for a relative temporal sequence of impulse sequences produced. ### Claim 4 (depends on 2) The prosthesis according to claim 2, wherein a monitor is provided that displays imaging functions produced by the RF filters at least approximately inverted and optically or acoustically. ### Claim 5 (depends on 1) The prosthesis according to claim 1, further comprising: a simulated eye movement system, including head and eye movement detectors. ### Claim 6 (depends on 1) The prosthesis according to claim 1, wherein said implantable microstructure comprises: a device for application of an active substance, the device being controlled by said central control unit; wherein the implantable microstructure records and transmits neural activity, in the form of said monitoring signals, to a monitoring system of said central control unit. ### Claim 7 (depends on 1) The prosthesis according to claim 1, wherein said central control unit further comprises: an adaptive pre-programming module, for simplification of image or sound patterns. ### Claim 8 (depends on 1) The prosthesis according to claim 1, further comprising: a portable signal transmitter for relaying a position of an object in space, determined by the encoder, to an appropriate sensory organ. ### Claim 9 (depends on 1) The prosthesis according to claim 1, wherein said central control unit includes at least one pattern recognition program. ### Claim 10 (depends on 1) The prosthesis according to claim 1, further comprising: feedback apparatus for training said encoder. ### Claim 11 (depends on 10) The prosthesis according to claim 10, wherein said feedback apparatus comprises: means for inputting evaluation inputs, the evaluation inputs representing a comparison between an original stimulus and a detected stimulus; and a dialog module, the dialog module receiving the evaluation inputs and providing feedback to the encoder for purposes of adaptation. ### Claim 12 (depends on 11) The prosthesis according to claim 11, wherein said dialog module comprises a neural network. ### Claim 13 An adaptive sensory-motor encoder for use in a visual or acoustic prosthesis, the prosthesis including an implantable microstructure for stimulation of neural or glial tissue and for monitoring of neural function and a bi-directional interface coupled to the implantable microstructure, the bi-directional interface facilitating the transmission of at least one of stimulation and control signals from the encoder to the microstructure and of monitoring signals from the microstructure to the encoder, the encoder comprising: a central control unit performing signal processing functions, monitoring functions, control functions, and external pick-up functions, wherein the central control unit includes a group of adaptive spatio-temporal filters, the adaptive spatio-temporal filters converting sensory signals into stimulation pulse sequences. ### Claim 14 (depends on 13) The encoder according to claim 13, wherein the control unit includes in a perception-based dialog process receptive field (RF) filters, said RF filters comprising adjustable spatio-temporal filters with receptive field properties. ### Claim 15 (depends on 14) The encoder according to claim 14, wherein the RF filters are associated with time delay elements for a relative temporal sequence of impulse sequences produced. ### Claim 16 (depends on 14) The encoder according to claim 14, wherein a monitor is provided that displays imaging functions produced by the RF filters at least approximately inverted and optically or acoustically. ### Claim 17 (depends on 13) The encoder according to claim 13, further comprising: a simulated eye movement system, the simulated eye movement system receiving input from head and eye movement detectors. ### Claim 18 (depends on 13) The encoder according to claim 13, wherein said central control unit controls a device for application of an active substance, the device being part of said implantable microstructure; and wherein the central control unit includes a monitoring system that receives indications of neural activity from said implantable microstructure via said monitoring signals. ### Claim 19 (depends on 13) The encoder according to claim 13, wherein said central control unit further comprises: an adaptive pre-programming module, for simplification of image or sound patterns. ### Claim 20 (depends on 13) The encoder according to claim 13, wherein said central control unit includes at least one pattern recognition program. ### Claim 21 A method for use with an adaptive sensory-motor encoder for a visual or acoustic prosthesis, the encoder comprising a central control unit performing signal processing functions, monitoring functions, control functions and external pick-up functions, as well as including a group of adaptive spatio-temporal filters, the prosthesis further including a microstructure for stimulation and monitoring of neural activity and a bi-directional interface coupling the encoder with the microstructure for transmission of at least one of control and stimulation signals from the encoder to the microstructure and of monitoring signals from the microstructure to the encoder, the method comprising the steps of: supplying a minimum single-channel evaluation entry unit for individual adjustment of said signal processing functions by means of a dialog process, the evaluation entry unit using a subjective evaluation vector, where the evaluation vector represents similarity of a currently perceived pattern to a desired pattern; transmitting the evaluation vector to a parameter adjustment system, to produce suitable sequences of parameter vectors, said parameter adjustment system comprising a neural network with non-monitored adaptation rules; and generating, by said evaluation entry unit, a parameter vector at a multi-channel output for a respective signal processing function to be adjusted. ### Claim 22 (depends on 21) The method according to claim 21, wherein said signal processing functions comprise receptive field (RF) filters. ### Claim 23 (depends on 22) The method according to claim 22, further comprising the step of: establishing, by the evaluation entry unit, appropriate pattern sequences for an adaptation process in a decision system for individual or group optimization of said RF filters. ### Claim 24 (depends on 23) The method according to claim 23, wherein said step of establishing comprises the step of: generating internally stored sequences of parameter vectors for establishing typical RF filter functions. ### Claim 25 (depends on 24) The method according to claim 24, wherein said RF filters have a spatio-temporal function space that includes a function space of the receptive field properties of neurons at a site at which said microstructure is implanted. ### Claim 26 (depends on 22) The method according to claim 22, further comprising the steps of: generating at least one signal by the RF filters; emitting said at least one signal at said microstructure to elicit an actual perception; and simultaneously passing an associated desired pattern to an output unit for perception by a human. ### Claim 27 (depends on 26) The method according to claim 26, wherein said microstructure includes a given number of micro-contacts, and further comprising the step of: conducting, by an RF filter, said at least one signal to several locally adjacent micro-contacts for functionally increasing the number and definition of selectively reachable stimulation sites. ### Claim 28 (depends on 26) The method according to claim 26, wherein said at least one signal comprises at least two impulse signals, and further comprising the step of: varying the at least two impulse signals for the purpose of shifting a focus of stimulation and varying stimulation sites. ### Claim 29 (depends on 28) The method according to claim 28, wherein said step of varying the at least two impulse signals comprises the step of: effecting a perception-based dialog with a being in which the microstructure is implanted, for facilitating shifting of the stimulation focus so as to lead to selective and well-defined neural stimulation, the step of effecting a perception-based dialog including a step of utilizing optimization means in the central control unit. ### Claim 30 (depends on 29) The method according to claim 29, wherein said step of utilizing optimization means includes the step of utilizing a neural network. ### Claim 31 (depends on 26) The method according to claim 26, wherein said step of generating at least one signal comprises the step of generating at least one impulse signal. ### Claim 32 (depends on 31) The method according to claim 31, further comprising the steps of: comparing recorded neural impulses to said at least one impulse signal; distinguishing spontaneously occurring neural impulses from those produced in response to said at least one impulse signal; and improving selectivity and biocompatibility of neural stimulation effected by said at least one impulse signal using results of said steps of comparing and distinguishing. ### Claim 33 (depends on 21) The method according to claim 21, further comprising the steps of: detecting head movements and undesirable real eye movements; simulating eye movements by means of at least one of electronic image pattern shifts, optical variation of direction of vision, and movement of photosensors; using the detected head and real eye movements, as well as simulated eye movements produced by said step of simulating eye movements, by means of movement control or adjustment, quick and slow eye movements are produced for purposes of pattern recognition, tracking of moving objects, and fast circumspection. ### Claim 34 (depends on 33) The method of claim 33, further comprising the step of: using said quick and slow eye movements to compensate for undesired eye movements or perceptions of apparent motion. ### Claim 35 (depends on 33) The method of claim 33, further comprising the step of: generating compensatory eye movements with reference to the human vestibulo-ocular reflex for situational stabilization of an image pattern in the presence of normal head and upper body movements. ### Claim 36 (depends on 33) The method of claim 33, further comprising the step of: facilitating technical adaptation of a brightness operating range resulting from a function range of a photosensor array extending over several brightness decades, said step of facilitating including the step of: selecting an operating range for the encoder with respect to magnitude and adaptation brightness. ### Claim 37 (depends on 36) The method of claim 36, wherein said step of selecting an operating range comprises the steps of: non-linearly evaluating sections of said function range; and composing said operating range from the resulting non-linearly evaluated sections of the function range. ### Claim 38 (depends on 37) The method of claim 37, further comprising the step of: transposing visual scenes with local subsets of varying brightness into a common operating range of the encoder, using a suitable imaging system. ### Claim 39 (depends on 37) The method of claim 37, further comprising the step of: if necessary, shifting RF filter functions for stimulation of perception a bright or dark adapted range using a brightness operating range of the encoder. ### Claim 40 (depends on 39) The method of claim 39, further comprising the step of: rapidly changing the brightness operating range and the associated RF filter function shift automatically during simulated eye movements or during pattern recognition. ### Claim 41 (depends on 36) The method of claim 36, further comprising the steps of: sharply adjusting first areas of a visual field by variation of accommodation; storing the resulting sharply-adjusted first areas; sharply adjusting, as an image, second areas of said visual field, whereby said initial areas become less sharp; and blending the stored sharply-adjusted first areas into the image in place of the first areas, which became less sharp in said step of sharply adjusting second areas. ### Claim 42 (depends on 36) The method of claim 36, wherein the method repeats itself cyclically. ### Claim 43 (depends on 21) The method of claim 21, further comprising the step of: relaying a position of an object in space, determined by said encoder, to an appropriate sensory organ using a portable signal transmitter. ### Claim 44 (depends on 21) The method of claim 21, further comprising the steps of: executing, in the encoder, at least one pattern recognition program, in connection with automatically running, simulated eye movements; warning an implant carrier of obstacles or hazards detected based on results of said step of executing; and reporting a type and position of a technically identified pattern or object detected based on results of said step of executing.