# US 8,315,305 B2 · Systems and methods for invariant pulse latency coding

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 8,315,305 B2
- Document ID: US8315305B2
- Status: Granted
- Kind: B2
- Application number: 12/869,573
- Filed: 2010-08-26
- Published: 2011-09-29
- Granted: 2012-11-20
- Earliest claimed: 2010-03-26
- Assignees: Brain Corporation
- Inventors: Csaba Petre; Botond Szatmary; Eugene M. Izhikevich
- Web page: https://patentgrep.com/patent/US8315305B2

## CPC

- H04N 19/00: Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- G06N 3/049: Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
- G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51)
- G06T 7/40: Analysis of texture (depth or shape recovery from texture G06T7/529)
- G06T 9/002: {using neural networks}
- G06V 10/40: Extraction of image or video features
- H04B 14/026: {using pulse time characteristics modulation, e.g. width, position, interval}
- H04N 19/107: between spatial and temporal predictive coding, e.g. picture refresh
- G06T 2207/10016: Video; Image sequence
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04B: TRANSMISSION

## Abstract

Image processing systems and methods extract information from an input signal representative of an element of an image and to encode the information in a pulsed output signal. A plurality of channels communicates the pulsed output signal, each of the plurality of channels being characterized by a latency. The information may be encoded as a pattern of relative pulse latencies observable in pulses communicated through the plurality of channels and the pattern of relative pulse latencies is substantially insensitive to image contrast and/or image luminance. A filter can be employed to provide a generator signal based on the input signal and pulse latencies can be determined using a logarithmic function of the generator signal. The filter may be temporally and/or spatially balanced and characterized by an integral along spatial and/or temporal dimensions of the filter that is substantially zero for all values of a temporal and/or a spatial variable.

## Claims

### Claim 1

An image processing system, the system comprising:
an input signal conveying information associated with an element of the image;
one or more processors configured to encode the information in a pattern of relative pulse latencies communicated via a plurality of channels; and
a filter configured to provide a generator signal based on the input signal such that:
g(t)=LM∫∫S(x,t−s)F(x,s)dxds,
where g(t) represents the generator signal as a function of time, L represents an image luminance, M represents an image contrast, S(x, t−s) represents a feature in the image as a function of space and time, and F(x, s) represents a filter kernel as a function of space and time;
wherein the pattern of relative pulse latencies is insensitive to one or both of the image contrast or the image luminance.

### Claim 2 (depends on 1)

The system of claim 1, wherein the pulse latencies are determined using a logarithmic function of the generator signal.

### Claim 3 (depends on 2)

The system of claim 2, wherein the logarithmic function is applied to a rectified version of the generator signal.

### Claim 4 (depends on 2)

The system of claim 2, wherein:
the image is characterized by a spatial dimension; and
the filter kernel comprises a spatially balanced kernel having an integral of the filter kernel along the spatial dimensions that is zero for one or more values of a temporal variable.

### Claim 5 (depends on 2)

The system of claim 2, wherein:
the image is characterized by a spatial dimension; and
the filter kernel is temporally balanced by an integral of the filter kernel along a temporal dimension of the filter that is zero for individual values of a spatial variable along the spatial dimension.

### Claim 6 (depends on 2)

The system of claim 2, wherein the filter is balanced by an integral along one or more dimensions of the filter equal to zero.

### Claim 7 (depends on 1)

The system of claim 1, wherein individual ones of the pulse latencies include a common latency associated with the image contrast.

### Claim 8 (depends on 1)

The system of claim 1, further comprising a decoder that decodes a pulsed output signal based on the relative pulse latencies.

### Claim 9 (depends on 8)

The system of claim 8, wherein the decoder receives the pulsed output signal from the plurality of channels.

### Claim 10 (depends on 9)

The system of claim 9, wherein the decoder detects coincident arrival of pulses on different ones of the plurality of channels.

### Claim 11 (depends on 9)

The system of claim 9, wherein the decoder produces secondary information and encodes the secondary information into the timing of pulses in a decoder output.

### Claim 12 (depends on 8)

The system of claim 8, wherein the decoder receives the pulsed output signal from the plurality of channels and at least two of the plurality of channels have different transmission delays.

### Claim 13

An image processing system, the system comprising:
one or more processors configured to (i) extract information from an input signal conveying information associated with an element of the image and (ii) encode the information in a pattern of relative pulse latencies; and
a filter configured to provide a generator signal based on the input signal;
wherein a given latency in the pattern of relative pulse latencies is defined by:
Lat=C−logB|g(t)/a(t)|+,
where Lat represents the given latency, C represents an offset, B represents a base of the logarithmic function, g(t) represents the generator signal as a function of time, and a(t) represents an adaptation variable as a function of time.

### Claim 14 (depends on 13)

The system of claim 13, wherein the filter comprises a linear filter.

### Claim 15 (depends on 13)

The system of claim 13, wherein the offset and the base of the logarithmic function are optimized to obtain a range of the generator signal.

### Claim 16 (depends on 15)

The system of claim 15, wherein the range is set to match the dynamic range of the latency values and the dynamic range of the input signal.

### Claim 17 (depends on 13)

The system of claim 13, wherein pulsed outputs are generated in a pulsed output signal responsive to an occurrence of one or more of a cyclic event, arrival of an input frame, appearance of a new feature in the image, or a time related to a previous event.

### Claim 18 (depends on 17)

The system of claim 17, wherein the image processing system is at least partially embodied in a prosthetic device.

### Claim 19 (depends on 17)

The system of claim 17, wherein the image processing system is at least partially embodied in an autonomous robot.

### Claim 20 (depends on 17)

The system of claim 17, wherein the image processing system is at least partially embodied in a remote server.

### Claim 21 (depends on 13)

The system of claim 13, further comprising bounding the latencies of the pulses within a selected interval.

### Claim 22 (depends on 13)

The system of claim 13, wherein the plurality of channels comprises one or more virtual channels carried in a physical transmission medium.

### Claim 23 (depends on 13)

The system of claim 13, wherein the filter comprises a non-linear filter.

### Claim 24 (depends on 23)

The system of claim 23, wherein the filter is motion sensitive.

### Claim 25 (depends on 24)

The system of claim 24, wherein the filter is direction sensitive.

### Claim 26

A method of processing an input signal conveying information associated with an image, the method being performed by one or more processors configured to execute computer program modules, the method comprising:
based on the input signal, determining a generator signal using a filter such that:
g(t)=LM∫∫S(x,t−s)F(x,s)dxds,
where g(t) represents the generator signal as a function of time, L represents image luminance, M represents the image contrast, S(x, t−s) represents a feature in the image as a function of space and time, and F(x, s) represents filter kernel as a function of space and time; and
based on the generator signal, encoding the information in a pattern of relative pulse latencies;
wherein the pattern of relative pulse latencies is insensitive to one or both of image contrast or image luminance.

### Claim 27 (depends on 26)

The method of claim 26, wherein the pulse latencies are determined using a logarithmic function of the generator signal.