# US 12,003,874 B2 · Image sensors and sensing methods to obtain Time-of-Flight and phase detection information

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,003,874 B2
- Document ID: US12003874B2
- Status: Granted
- Kind: B2
- Application number: 18/450,436
- Filed: 2023-08-16
- Published: 2024-01-04
- Granted: 2024-06-04
- Earliest claimed: 2020-07-15
- Assignees: Corephotonics Ltd.
- Applicants: Corephotonics Ltd.
- Inventors: Nadav Geva; Michael Scherer; Ephraim Goldenberg; Gal Shabtay
- Web page: https://patentgrep.com/patent/US12003874B2

## CPC

- H04N 13/232: using fly-eye lenses, e.g. arrangements of circular lenses
- H04N 25/705: Pixels for depth measurement, e.g. RGBZ
- G01S 7/4816: {of receivers alone}
- G01S 17/894: Three-dimensional [3D] imaging with simultaneous measurement of time-of-flight at a two-dimensional [2D] array of receiver pixels, e.g. time-of-flight cameras or flash lidar
- G06T 7/593: from stereo images
- H04N 13/128: Adjusting depth or disparity
- H04N 13/207: using a single two-dimensional [2D] image sensor
- H04N 13/254: in combination with electromagnetic radiation sources for illuminating objects
- H04N 13/271: wherein the generated image signals comprise depth maps or disparity maps
- H04N 25/75: Circuitry for providing, modifying or processing image signals from the pixel array
- H04N 13/122: Improving the three-dimensional [3D] impression of stereoscopic images by modifying image signal contents, e.g. by filtering or adding monoscopic depth cues (H04N13/128 takes precedence)
- H04N 2013/0081: {Depth or disparity estimation from stereoscopic image signals}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Indirect time-of-flight (i-ToF) image sensor pixels, i-ToF image sensors including such pixels, stereo cameras including such image sensors, and sensing methods to obtain i-ToF detection and phase detection information using such image sensors and stereo cameras. An i-ToF image sensor pixel may comprise a plurality of sub-pixels, each sub-pixel including a photodiode, a single microlens covering the plurality of sub-pixels and a read-out circuit for extracting i-ToF phase signals of each sub-pixel individually.

## Claims

### Claim 1

An image sensor pixel, comprising:
a plurality of sub-pixels;
a microlens covering the plurality of sub-pixels; and
a read-out circuit (ROC) for extracting time-of-flight (ToF) signals of each sub-pixel individually, wherein the image sensor pixel includes a switch, wherein in one state the switch is closed so that the plurality of sub-pixels form together one pixel and the ROC reads out the one pixel for generating a ToF depth map, and wherein in another state the switch is opened so that the ROC reads out each sub-pixel individually for generating a stereo depth map.

### Claim 2 (depends on 1)

The image sensor pixel of claim 1, wherein the plurality of sub-pixels includes 2 sub-pixels.

### Claim 3 (depends on 1)

The image sensor pixel of claim 1, wherein the plurality of sub-pixels includes 4 sub-pixels.

### Claim 4 (depends on 1)

The image sensor pixel of claim 1, wherein the image sensor pixel is a direct ToF (d-ToF) image sensor pixel.

### Claim 5 (depends on 1)

The image sensor pixel of claim 1, wherein the image sensor pixel is an indirect ToF (i-ToF) image sensor pixel.

### Claim 6 (depends on 1)

The image sensor pixel of claim 1, included in an image sensor of a camera having a focal length fin the range of 1.5 mm-10 mm.

### Claim 7 (depends on 1)

The image sensor pixel of claim 1, included in an image sensor of a stereo camera having a baseline B in the range of 0.5 mm-50 mm.

### Claim 8 (depends on 1)

The image sensor pixel of claim 1, included in an image sensor of a stereo camera having a baseline B in the range of 0.5 mm-25 mm.

### Claim 9 (depends on 1)

The image sensor pixel of claim 1, included in an image sensor of a stereo camera having a baseline B in the range of 0.5 mm-10 mm.

### Claim 10 (depends on 1)

The image sensor pixel of claim 1, wherein each sub-pixel has a size of 1 μm-10 μm.

### Claim 11 (depends on 1)

The image sensor pixel of claim 1, included in an image sensor of a stereo camera that has a vertical or a horizontal baseline.

### Claim 12 (depends on 1)

The image sensor pixel of claim 1, wherein the image sensor pixel is included in a image sensor of a stereo camera, and wherein the stereo camera is included in a mobile device.

### Claim 13 (depends on 12)

The image sensor pixel of claim 12, wherein the mobile device is a smartphone.

### Claim 14

A method, comprising:
providing a time-of-flight (ToF) image sensor pixel that includes a plurality of sub-pixels, wherein the plurality of sub-pixels is covered by a microlens;
using a read-out circuit (ROC) to read out ToF signals of each sub-pixel individually, by:
obtaining a stereo depth map and an ToF map calculated from the ToF signals;
analyzing the stereo depth map and the ToF depth map for assigning stereo scores to segments of the stereo depth map and for assigning ToF scores to the segments of the ToF depth map; and
generating a fused depth map by using stereo depth map data for segments that have high stereo scores and using ToF depth map data for segments that have high ToF scores.

### Claim 15 (depends on 14)

The method of claim 14, wherein the using a ROC to read out ToF signals of each sub-pixel individually includes using the ROC to read out fewer than all ToF signals generated by the plurality of the sub-pixels.

### Claim 16 (depends on 14)

The method of claim 14, wherein the image sensor pixel is a direct ToF (d-ToF) image sensor pixel.

### Claim 17 (depends on 14)

The method of claim 14, wherein the image sensor pixel is an indirect ToF (i-ToF) image sensor pixel.

### Claim 18 (depends on 14)

The method of claim 14, wherein the 1-shot depth map is used to generate a high fps depth map stream having a fps≥35.

### Claim 19 (depends on 14)

The method of claim 14, wherein the image sensor pixel is included in a image sensor of a stereo camera, and wherein the stereo camera is included in a mobile device.

### Claim 20 (depends on 19)

The method of claim 19, wherein the mobile device is a smartphone.