# US 2016/0364875 A1 · Image processing device, image processing method, and information storage device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2016/0364875 A1
- Document ID: US20160364875A1
- Status: Published application
- Kind: A1
- Application number: 15/248,780
- Filed: 2016-08-26
- Published: 2016-12-15
- Earliest claimed: 2014-02-28
- Assignees: Olympus Corporation
- Applicants: Olympus Corporation
- Inventors: Asako Seki
- Web page: https://patentgrep.com/patent/US20160364875A1

## CPC

- G06T 7/593: from stereo images
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
- G06T 2207/10021: Stereoscopic video; Stereoscopic image sequence
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/20048: Transform domain processing
- G06T 2207/20228: Disparity calculation for image-based rendering
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

An image processing device includes a processor including hardware, the processor being configured to calculate comparison result information through a comparison process performed on the results of a census transform process performed on a first image and the results of the census transform process performed on a second image, set weight information, calculate a cost using the comparison result information and the weight information, calculate the amount of disparity using the cost, set a first weight having as low degree of contribution to the cost on a neighboring pixel which is determined to be affected by noise to a large extent among a plurality of neighboring pixels, and set a second weight having a high degree of contribution to the cost on a neighboring pixel which is determined to be affected by noise to a small extent among the plurality of neighboring pixels.

## Claims

### Claim 1

An image processing device comprising:
a processor comprising hardware, the processor being configured to:
during a stereo matching process performed on the first image and the second image,
perform a census transform process on a first image and a second image, and calculate comparison result information through a comparison process between results of the census transform process on the first image and results of the census transform process on the second image;
set weight information for calculating a cost;
calculate the cost at a pixel of interest using the comparison result information and the weight information; and
calculate an amount of disparity at the pixel of interest using the cost,
wherein the processor is configured to set a first weight having a low degree of contribution to the cost on a neighboring pixel which is determined to be affected by noise to a large extent among a plurality of neighboring pixels situated around the pixel of interest, and set a second weight having a high degree of contribution to the cost on a neighboring pixel which is determined to be affected by the noise to a small extent among the plurality of neighboring pixels, the plurality of neighboring pixels being situated in a given area within the first image and the second image.

### Claim 2 (depends on 1)

The image processing device as defined in claim 1,
wherein the processor is configured to calculate the cost referring to a conversion table that links the comparison result information, the weight information, and the cost.

### Claim 3 (depends on 1)

The image processing device as defined in claim 1,
wherein the processor is configured to calculate a differential value between the pixel of interest and the neighboring pixel, and set the weight information by determining an effect of the noise from the calculated differential value.

### Claim 4 (depends on 3)

The image processing device as defined in claim 3,
wherein the processor is configured to set the weight information that is set on each neighboring pixel based on a result obtained by comparing the differential value between the neighboring pixel and the pixel of interest in the first image, with a given threshold value, and a result obtained by comparing the differential value between the neighboring pixel and the pixel of interest in the second image, with the given threshold value.

### Claim 5 (depends on 4)

The image processing device as defined in claim 4,
wherein the processor is configured to set the given threshold value that is compared with the differential value corresponding to the noise value.

### Claim 6 (depends on 4)

The image processing device as defined in claim 4,
wherein the processor is configured to set the first weight on the neighboring pixel when at least one of the differential values, which are between the neighboring pixel and the pixel of interest, in the first image and the second image, is equal to or smaller than the given threshold value.

### Claim 7 (depends on 1)

The image processing device as defined in claim 1,
wherein the processor is configured to convert results of the comparison process into numerical values to acquire the comparison result information.

### Claim 8 (depends on 1)

The image processing device as defined in claim 1,
wherein the processor is configured to set a weight mask as the weight information, the weight mask having a configuration in which a first logic level is set as the first weight, and a second logic level that differs from the first logic level is set as the second weight.

### Claim 9 (depends on 8)

The image processing device as defined in claim 8,
wherein the processor is configured to calculate first weight information on the first image by setting the first logic level as the first weight and setting the second logic level as the second weight, calculate second weight information on the second image by setting the first logic level as the first weight, and setting the second logic level as the second weight, and perform a logical calculation using the first weight information and the second weight information to calculate the weight mask.

### Claim 10 (depends on 8)

The image processing device as defined in claim 8,
wherein the processor is configured to skip the calculating of the cost with respect to a pixel at which a value of the weight mask is set to the first logic level.

### Claim 11 (depends on 8)

The image processing device as defined in claim 8,
wherein the processor is configured to refer to a conversion table in which a conversion result is determined corresponding to a combination of a value represented by the comparison result information and a value of the weight mask, and calculate the cost using the conversion result calculated on each neighboring pixel within the given area.

### Claim 12 (depends on 8)

The image processing device as defined in claim 8,
wherein the processor is configured to calculate the cost by calculating average using the result of cumulative sum, which is by calculating on each neighboring pixel using the comparison result information and the weight mask, and a number of the neighboring pixels set to the second logic level.

### Claim 13

An image processing method comprising:
during a stereo matching process performed on the first image and the second image,
performing a census transform process on a first image and a second image, and calculating comparison result information through a comparison process between results of the census transform process on the first image and results of the census transform process on the second image;
setting weight information for calculating a cost;
calculating the cost at a pixel of interest using the comparison result information and the weight information; and
calculating an amount of disparity at the pixel of interest using the cost,
wherein the setting of the weight information includes setting a first weight having a low degree of contribution to the cost on a neighboring pixel which is determined to be affected by noise to a large extent among a plurality of neighboring pixels situated around the pixel of interest, and setting a second weight having a high degree of contribution to the cost on a neighboring pixel which is determined to be affected by the noise to a small extent among the plurality of neighboring pixels, the plurality of neighboring pixels being situated in a given area within the first image and the second image.

### Claim 14

A computer-readable storage device with an executable program stored thereon, wherein the program instructs a microprocessor to perform the following steps of:
during a stereo matching process performed on the first image and the second image,
performing a census transform process on a first image and a second image and calculating comparison result information through a comparison process between results of the census transform process on the first image and results of the census transform process on the second image;
setting weight information for calculating a cost;
calculating the cost at a pixel of interest using the comparison result information and the weight information; and
calculating an amount of disparity at the pixel of interest using the cost,
wherein the setting of the weight information includes setting a first weight having a low degree of contribution to the cost on a neighboring pixel which is determined to be affected by noise to a large extent among a plurality of neighboring pixels situated around the pixel of interest, and setting a second weight having a high degree of contribution to the cost on a neighboring pixel which is determined to be affected by the noise to a small extent among the plurality of neighboring pixels, the plurality of neighboring pixels being situated in a given area within the first image and the second image.
