# US 10,504,235 B2 · Method for generating three dimensional images

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,504,235 B2
- Document ID: US10504235B2
- Status: Granted
- Kind: B2
- Application number: 15/391,267
- Filed: 2016-12-27
- Published: 2018-06-28
- Granted: 2019-12-10
- Earliest claimed: 2016-12-27
- Assignees: Inuitive Ltd.
- Applicants: Inuitive Ltd.
- Inventors: Lev Goldentouch; Ziv Tsoref; Samuel Lifsches
- Web page: https://patentgrep.com/patent/US10504235B2

## CPC

- G06T 7/593: from stereo images
- H04N 13/00: Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N 13/111: Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
- H04N 13/128: Adjusting depth or disparity
- H04N 13/207: using a single two-dimensional [2D] image sensor
- H04N 13/239: using two two-dimensional [2D] image sensors having a relative position equal to or related to the interocular distance (H04N13/243 takes precedence)
- G06T 2207/10012: Stereo images
- G06T 2207/10024: Color image
- G06T 2207/20076: Probabilistic image processing
- H04N 2013/0081: {Depth or disparity estimation from stereoscopic image signals}
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A method is provided for reconstructing a three dimensional image, comprising: providing a pair of stereoscopic images captured by two image capturing devices; retrieving disparity data from the pair of stereoscopic images; using retrieved disparity data to determine contours of an object present in the pair of the stereoscopic images captured; selecting a pixel belonging to that object; establishing a search area at the vicinity of the selected pixel; determining values for various disparity candidates, wherein the determined values correspond to one or more chosen hypothesis associated with the search area; testing the one or more chosen hypothesis and selecting an hypothesis which best fits the selected pixel and its neighboring pixels; executing a stereo matching algorithm using data retrieved from the captured images while using the values of disparity candidates that correspond to the chosen hypothesis; and reconstructing the three dimensional image based on the results obtained.

## Claims

### Claim 1

A method for reconstructing a three dimensional image, said method comprises the steps of:
receiving a pair of stereoscopic images captured essentially simultaneously by two image capturing devices;
retrieving disparity data from said pair of stereoscopic images and calculating disparity edges therefrom based on intensity gradients that exist between pixels comprised in said stereoscopic images;
using the calculated disparity edges to determine contours of an object present in the pair of the stereoscopic images captured;
selecting at least one pixel belonging to said object;
establishing a search area at the vicinity of the at least one selected pixel to enable identifying disparity candidates associated with objects located at the vicinity of said object whose contours were determined, wherein a size of the search area is adapted to a disparity level of the selected at least one pixel;
determining values for the identified disparity candidates, wherein the determined values correspond to one or more chosen hypothesis associated with said search area;
testing the one or more chosen hypothesis and selecting a hypothesis which best fits the at least one selected pixel and its neighboring pixels;
executing a stereo matching algorithm using data retrieved from the pair of corresponding captured images while using the values of one or more identified disparity candidates that correspond to the chosen hypothesis; and
reconstructing the three dimensional image based on the results of the stereo matching algorithm.

### Claim 2 (depends on 1)

The method of claim 1, wherein said method comprises an iterative process which comprises receiving data that relates to disparity candidates obtained from a low-noise iteration and executing a new high-resolution iteration.

### Claim 3 (depends on 1)

The method of claim 1, wherein said method comprises retrieving data that relates to disparity candidates over a group of pixels present in the area located at the vicinity of the at least one selected pixel.

### Claim 4 (depends on 3)

The method of claim 3, further comprising a step of selecting a subset of disparity candidates from among the retrieved data that relates to said disparity, for re-evaluating said at least one selected pixel.

### Claim 5 (depends on 1)

The method of claim 1, wherein said step of testing the one or more chosen hypothesis comprises using a high-resolution adaptive block matching of the area located around the at least one selected pixel.

### Claim 6 (depends on 1)

The method of claim 1, wherein said objects located at the vicinity of the object whose contour was determined are present at different depths of the image.

### Claim 7 (depends on 1)

The method of claim 1, wherein said disparity edges are calculated by using an edge detector or by taking into account changes that occurred in said disparity data that exceed a pre-determined threshold, or any combination thereof.

### Claim 8 (depends on 1)

The method of claim 1, wherein said disparity edges are calculated by using a discrete differentiation operator for computing an approximation of intensities' gradients in the stereoscopic images.

### Claim 9

A system for reconstructing a three dimensional image, the system comprising:
at least two image capturing devices for capturing essentially simultaneously at least two stereoscopic images;
at least one processor configured to:
retrieve disparity data from said at least two stereoscopic images;
calculating disparity edges therefrom based on intensity gradients that exist between pixels comprised in said stereoscopic images;
apply retrieved the calculated disparity edges to determine contours of an object present in the at least two stereoscopic images captured;
select at least one pixel belonging to the object;
establish a search area at the vicinity of the at least one selected pixel to enable identifying disparity candidates associated with objects located at the vicinity of said object whose contours were determined, wherein a size of the search area is adapted to a disparity level of the selected at least one pixel;
determine values for the identified disparity candidates, wherein the determined values correspond to one or more chosen hypothesis associated with said search area;
test the one or more chosen hypothesis and select a hypothesis which best describes the at least one selected pixel and its neighboring pixels;
execute a stereo matching algorithm using data retrieved from the at least two captured images while using the values of one or more identified disparity candidates that correspond to the chosen hypothesis; and
reconstruct the three dimensional image based on the results of the stereo matching algorithm;
a memory means configured for storing information retrieved from the at least two image capturing devices along with disparity information that relates to said search area; and
a delay buffer configured to enable synchronization between data received from the at least two image capturing devices.

### Claim 10 (depends on 9)

The system of claim 9, wherein the at least one processor is further configured to determine that disparity candidates available around the contours belong to different objects, while disparity candidates associated with pixels located relatively far from the determined contours belong to said object.

### Claim 11 (depends on 10)

The system of claim 10, wherein the at least one processor is further configured to establish a smaller search area for the pixels located relatively far from the determined contours.

### Claim 12 (depends on 9)

The system of claim 9, wherein the size of the search area is different than a size of an area used for computing low-noise disparity.

### Claim 13 (depends on 9)

The system of claim 9, wherein the at least one processor is further configured to determine depth from the at least two captured images.

### Claim 14 (depends on 9)

The system of claim 9, wherein the at least one processor is further configured to determine a depth of the three dimensional image by which low-noise disparity candidates are selected over a certain area for a secondary high-resolution disparity search carried out over selected candidates.

### Claim 15 (depends on 9)

The system of claim 9, wherein the at least one processor is further configured to determine a depth of the three dimensional image by selecting a disparity calculating mechanism which is adapted to enable balancing between noise level and resolution level according to user preferences.