# US 2020/0404246 A1 · Time-of-flight image sensor resolution enchantment and increased data robustness using a binning module

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0404246 A1
- Document ID: US20200404246A1
- Status: Published application
- Kind: A1
- Application number: 16/449,665
- Filed: 2019-06-24
- Published: 2020-12-24
- Earliest claimed: 2019-06-24
- Assignees: Infineon Technologies AG
- Applicants: Infineon Technologies AG
- Inventors: Krum BESHINSKI
- Web page: https://patentgrep.com/patent/US20200404246A1

## CPC

- H04N 13/254: in combination with electromagnetic radiation sources for illuminating objects
- G01S 17/894: Three-dimensional [3D] imaging with simultaneous measurement of time-of-flight at a two-dimensional [2D] array of receiver pixels, e.g. time-of-flight cameras or flash lidar
- H04N 13/271: wherein the generated image signals comprise depth maps or disparity maps
- H04N 25/46: by combining or binning pixels
- H04N 25/70: SSIS architectures; Circuits associated therewith
- H04N 25/78: Readout circuits for addressed sensors, e.g. output amplifiers or A/D converters
- G01S 7/4914: of detector arrays, e.g. charge-transfer gates
- G01S 7/493: Extracting wanted echo signals
- G01S 17/36: with phase comparison between the received signal and the contemporaneously transmitted signal
- G01S 17/48: Active triangulation systems, i.e. using the transmission and reflection of electromagnetic waves other than radio waves
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES

## Abstract

A time-of-flight (ToF) image sensor system includes a pixel array, where each pixel of the pixel array is configured to receive a reflected modulated light signal and to demodulate the reflected modulated light signal to generate an electrical signal; a plurality of analog-to-digital converters (ADCs), where each ADC is coupled to at least one assigned pixel of the pixel array and is configured to convert a corresponding electrical signal generated by the at least one assigned pixel into an actual pixel value; and a binning circuit coupled to the plurality of ADCs and configured to generate at least one interpolated pixel, where the binning circuit is configured to generate each of the at least one interpolated pixel based on actual pixel values corresponding to a different pair of adjacent pixels of the pixel array, each of the at least one interpolated pixel having a virtual pixel value.

## Claims

### Claim 1

A time-of-flight (ToF) image sensor system, comprising:
an illumination source configured to generate a modulated light signal;
a pixel array comprising a plurality of actual pixel lines and a plurality of actual pixel columns, wherein each pixel of the pixel array is configured to receive the modulated light signal reflected from an object as a reflected modulated light signal and to demodulate the reflected modulated light signal to generate an electrical signal;
a plurality of analog-to-digital converters (ADCs), wherein each ADC is coupled to at least one assigned pixel of the pixel array and is configured to convert a corresponding electrical signal generated by the at least one assigned pixel into an actual pixel value;
a binning circuit coupled to the plurality of ADCs and configured to generate at least one interpolated pixel, wherein the binning circuit is configured to generate each of the at least one interpolated pixel based on actual pixel values corresponding to a different pair of adjacent pixels of the pixel array, each of the at least one interpolated pixel having a virtual pixel value; and
an output circuit coupled to the binning circuit and configured to output the virtual pixel value of each of the at least one interpolated pixel according to an image structure.

### Claim 2 (depends on 1)

The ToF image sensor system of claim 1, wherein each virtual pixel value is an average of the actual pixel values corresponding to the different pair of adjacent pixels of the pixel array.

### Claim 3 (depends on 1)

The ToF image sensor system of claim 1, wherein:
the pixel array includes a first actual pixel line comprising a first plurality of pixels having first actual pixel values, wherein the first plurality of pixels is arranged into a first plurality of pairs of adjacent pixels, and
the binning circuit is configured to generate a first plurality of interpolated pixels having first virtual pixel values based on the first actual pixel values of the first plurality of pairs of adjacent pixels.

### Claim 4 (depends on 3)

The ToF image sensor system of claim 3, wherein the output circuit is configured to interleave the first actual pixel values with the first virtual pixel values in an image line of the image structure.

### Claim 5 (depends on 3)

The ToF image sensor system of claim 3, wherein:
the pixel array includes a second actual pixel line comprising a second plurality of pixels having second actual pixel values, wherein the second plurality of pixels is arranged into a second plurality of pairs of adjacent pixels, and
the binning circuit is configured to generate a second plurality of interpolated pixels having second virtual pixel values based on the second actual pixel values of the second plurality of pairs of adjacent pixels.

### Claim 6 (depends on 5)

The ToF image sensor system of claim 5, wherein the output circuit is configured to:
interleave the first actual pixel values with the first virtual pixel values in a first image line of the image structure, and
interleave the second actual pixel values with the second virtual pixel values in a second image line of the image structure.

### Claim 7 (depends on 5)

The ToF image sensor system of claim 5, wherein the output circuit is configured to:
output the first virtual pixel values in a first image line of the image structure, and
output the second virtual pixel values in a second image line of the the image structure.

### Claim 8 (depends on 7)

The ToF image sensor system of claim 7, wherein the image structure includes only virtual pixel values of interpolated pixels.

### Claim 9 (depends on 5)

The ToF image sensor system of claim 5, wherein:
the first actual pixel line is adjacent to the second actual pixel line such that the first plurality of pixels is combined with the second plurality of pixels into a third plurality of pairs of adjacent pixels, and
the binning circuit is configured to generate a third plurality of interpolated pixels having third virtual pixel values, each based on a different one of the first actual pixel values and a different one of the second actual pixel values.

### Claim 10 (depends on 9)

The ToF image sensor system of claim 9, wherein the output circuit is configured to:
output the first virtual pixel values in a first image line of the the image structure,
store the second virtual pixel values in a second image line of the image structure, and
store the third virtual pixel values in a third image line of the image structure, wherein the third image line is interposed between the first image line and the second image line.

### Claim 11 (depends on 10)

The ToF image sensor system of claim 10, wherein the image structure includes only virtual pixel values of interpolated pixels.

### Claim 12 (depends on 9)

The ToF image sensor system of claim 9, wherein:
the binning circuit is configured to generate a fourth plurality of interpolated pixels having fourth virtual pixel values, each based on a different one of the first virtual pixel values and a different one of the second virtual pixel values.

### Claim 13 (depends on 12)

The ToF image sensor system of claim 12, wherein the output circuit is configured to:
interleave the first actual pixel values with the first virtual pixel values in a first image line of the image structure,
interleave the second actual pixel values with the second virtual pixel values in a second image line of the image structure, and
interleave the third virtual pixel values with the fourth virtual pixel values in a third image line of the image structure.

### Claim 14 (depends on 13)

The ToF image sensor system of claim 13, wherein the third image line is interposed between the first image line and the second image line.

### Claim 15 (depends on 14)

The ToF image system of claim 14, wherein the image structure includes a plurality of virtual pixel columns each consisting of interpolated pixels.

### Claim 16 (depends on 1)

The ToF image sensor system of claim 1, wherein the binning circuit comprises:
a plurality of arithmetic logic units (ALUs) coupled to the plurality of ADCs and the output circuit, wherein each of the plurality of ALUs is coupled to a different pair of ADCs for receiving the actual pixel values therefrom and is configured to combine the actual pixel values to generate a different one of the at least one interpolated pixel.

### Claim 17 (depends on 1)

The ToF image sensor system of claim 1, wherein a total number of pixel values in the image structure is greater than a number of total pixels of the pixel array.

### Claim 18 (depends on 1)

The ToF image sensor system of claim 1, wherein:
the binning circuit is configured to receive a first number of actual pixel values from the plurality of ADCs, and
the output circuit is configured to output a second number of total pixel values according to the image structure, wherein the second number is greater than the first number.

### Claim 19 (depends on 18)

The ToF image sensor system of claim 18, wherein the second number of total pixel values include the first number of actual pixel values and a third number of virtual pixel values that includes each virtual pixel value of the at least one at least one interpolated pixel.

### Claim 20

A method of pixel interpolation in a time-of-flight (ToF) image sensor system, the method comprising:
generating a modulated light signal;
generating a plurality of electrical signals based on pixels of a pixel array receiving the modulated light signal reflected from an object as a reflected modulated light signal;
converting the plurality of electrical signals into a plurality of actual pixel values, wherein each actual pixel value of the plurality of actual pixel values corresponds to at least one pixel of the pixel array;
generating at least one interpolated pixel including generating each of the at least one interpolated pixel based on actual pixel values corresponding to a different pair of adjacent pixels of the pixel array, each of the at least one interpolated pixel having a virtual pixel value; and
outputting the virtual pixel value of each of the at least one interpolated pixel according to an image structure.

### Claim 21 (depends on 20)

The method of claim 20, wherein the pixel array includes a first actual pixel line comprising a first plurality of pixels having first actual pixel values, wherein the first plurality of pixels is arranged into a first plurality of pairs of adjacent pixels, and the method further comprises:
generating a first plurality of interpolated pixels having first virtual pixel values based on the first actual pixel values of the first plurality of pairs of adjacent pixels.

### Claim 22 (depends on 21)

The method of claim 21, wherein the pixel array includes a second actual pixel line comprising a second plurality of pixels having second actual pixel values, wherein the second plurality of pixels is arranged into a second plurality of pairs of adjacent pixels, and the method further comprises:
generating a second plurality of interpolated pixels having second virtual pixel values based on the second actual pixel values of the second plurality of pairs of adjacent pixels.

### Claim 23 (depends on 22)

The method of claim 22, further comprising:
interleaving the first actual pixel values with the first virtual pixel values in a first image line of the image structure; and
interleaving the second actual pixel values with the second virtual pixel values in a second image line of the image structure.

### Claim 24 (depends on 22)

The method of claim 22, further comprising:
outputting the first virtual pixel values in a first image line of the image structure; and
outputting the second virtual pixel values in a second image line of the image structure.
