# US 2017/0256067 A1 · Image processing device, image processing method, and solid-state imaging device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2017/0256067 A1
- Document ID: US20170256067A1
- Status: Published application
- Kind: A1
- Application number: 15/505,474
- Filed: 2015-08-21
- Published: 2017-09-07
- Earliest claimed: 2014-09-03
- Assignees: Sony Semiconductor Solutions Corporation
- Applicants: Sony Semiconductor Solutions Corporation
- Inventors: HIDESHI YAMADA
- Web page: https://patentgrep.com/patent/US20170256067A1

## CPC

- G06T 7/285: using a sequence of stereo image pairs
- G06T 5/50: using two or more images, e.g. averaging or subtraction
- G06T 5/92: based on global image properties
- G06T 7/223: using block-matching
- H04N 13/128: Adjusting depth or disparity
- H04N 13/189: Recording image signals; Reproducing recorded image signals
- H04N 13/239: using two two-dimensional [2D] image sensors having a relative position equal to or related to the interocular distance (H04N13/243 takes precedence)
- H04N 13/271: wherein the generated image signals comprise depth maps or disparity maps
- H04N 23/11: for generating image signals from visible and infrared light wavelengths
- H04N 23/62: Control of parameters via user interfaces
- H04N 23/6811: {based on the image signal}
- H04N 23/683: {performed by a processor, e.g. controlling the readout of an image memory}
- H04N 23/73: by influencing the exposure time
- H04N 23/741: by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors
- H04N 25/583: with different integration times
- G06T 2207/10016: Video; Image sequence
- G06T 2207/10021: Stereoscopic video; Stereoscopic image sequence
- G06T 2207/10144: Varying exposure
- G06T 2207/20021: Dividing image into blocks, subimages or windows
- G06T 2207/20208: High dynamic range [HDR] image processing
- G06T 2207/20221: Image fusion; Image merging
- G06T 2207/20224: Image subtraction
- H04N 2013/0081: {Depth or disparity estimation from stereoscopic image signals}
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

The present disclosure relates to an image processing device, an image processing method, and a solid-state imaging device capable of detecting image information with high accuracy by using images with different shooting conditions. A difference detection unit detects difference information of a short exposure image of a current frame and a past frame shot in a short exposure time. The difference detection unit detects difference information of a long exposure image of a current frame and a past frame shot in a long exposure time. The combining unit combines the difference information of the short exposure image and the difference information of the long exposure image on the basis of the short exposure image or the long exposure image and generates a motion vector of the current frame on the basis of the combined difference information. The present disclosure may be applied to the image processing apparatus and the like, for example.

## Claims

### Claim 1

An image processing device comprising:
a first difference detection unit which detects difference information of a first image of first and second frames shot with a first shooting condition;
a second difference detection unit which detects difference information of a second image of the first and second frames shot with a second shooting condition; and
a combining unit which combines the difference information of the first image detected by the first difference detection unit and the difference information of the second image detected by the second difference detection unit on the basis of the first or second image and generates a motion vector of the first frame on the basis of the combined difference information.

### Claim 2 (depends on 1)

The image processing device according to claim 1, wherein
the combining unit is configured to combine on the basis of the image with a better SN ratio out of the first and second images.

### Claim 3 (depends on 1)

The image processing device according to claim 1, wherein
the combining unit is configured to combine on the basis of luminance of the first or second image.

### Claim 4 (depends on 1)

The image processing device according to claim 1, wherein
the combining unit is configured to combine on the basis of contrast of the first or second image.

### Claim 5 (depends on 1)

The image processing device according to claim 1, wherein
the combining unit is configured to combine on the basis of the first or second image and a motion vector of a frame before the first frame.

### Claim 6 (depends on 5)

The image processing device according to claim 5, further comprising:
a blend ratio determination unit which determines a blend ratio of the difference information of the first image and the difference information of the second image on the basis of the first or second image and the motion vector of the frame before the first frame, wherein
the combining unit is configured to combine on the basis of the blend ratio determined by the blend ratio determination unit.

### Claim 7 (depends on 1)

The image processing device according to claim 1,
configured such that an exposure time, sensitivity, a shooting gain, or a focusing position is different between the first and second shooting conditions.

### Claim 8 (depends on 1)

The image processing device according to claim 1, further comprising:
a first motion detection unit which detects a first motion vector by using the first and second images of the first frame; and
a second motion detection unit which detects a second motion vector by using the first and second images of the second frame; wherein
the combining unit is configured to combine on the basis of the first or second image, the first motion vector, and the second motion vector.

### Claim 9

An image processing method comprising:
a first difference detection step of detecting difference information of a first image of first and second frames shot with a first shooting condition;
a second difference detection step of detecting difference information of a second image of the first and second frames shot with a second shooting condition; and
a combining step of combining the difference information of the first image detected by a process at the first difference detection step and the difference information of the second image detected by a process at the second difference detection step on the basis of the first or second image and generating a motion vector of the first frame on the basis of the combined difference information
by an image processing device.

### Claim 10

A solid-state imaging device comprising:
a shooting unit which shoots with first and second shooting conditions;
a first difference detection unit which detects difference information of a first image of first and second frames shot by the shooting unit with the first shooting condition;
a second difference detection unit which detects difference information of a second image of the first and second frames shot by the shooting unit with the second shooting condition; and
a combining unit which combines the difference information of the first image detected by the first difference detection unit and the difference information of the second image detected by the second difference detection unit on the basis of the first or second image and generates a motion vector of the first frame on the basis of the combined difference information.

### Claim 11

An image processing device comprising:
a first difference detection unit which detects difference information between a left-eye image and a right-eye image shot with a first shooting condition;
a second difference detection unit which detects difference information between the left-eye image and the right-eye image shot with a second shooting condition; and
a combining unit which combines the difference information with the first shooting condition detected by the first difference detection unit and the difference information with the second shooting condition detected by the second difference detection unit on the basis of the left-eye image shot with the first or second shooting condition or the right-eye image shot with the first or second shooting condition and generates the combined difference information as a depth value.

### Claim 12 (depends on 11)

The image processing device according to claim 11, wherein
the combining unit is configured to combine on the basis of the image with a better SN ratio out of the left-eye image shot with the first shooting condition and the left-eye image shot with the second shooting condition or the image with a better SN ratio out of the right-eye image shot with the first shooting condition and the right-eye image shot with the second shooting condition.

### Claim 13 (depends on 11)

The image processing device according to claim 11, wherein
the combining unit is configured to combine on the basis of luminance of the left-eye image shot with the first or second shooting condition or the right-eye image shot with the first or second shooting condition.

### Claim 14 (depends on 11)

The image processing device according to claim 11, wherein
the combining unit is configured to combine on the basis of contrast of the left-eye image shot with the first or second shooting condition or the right-eye image shot with the first or second shooting condition.

### Claim 15 (depends on 11)

The image processing device according to claim 11, wherein
the combining unit is configured to combine on the basis of the left-eye image shot with the first or second shooting condition or the right-eye image shot with the first or second shooting condition and a past depth value.

### Claim 16 (depends on 11)

The image processing device according to claim 11, wherein
the combining unit is configured to combine on the basis of the left-eye image shot with the first or second shooting condition or the right-eye image shot with the first or second shooting condition and a motion vector of the image.

### Claim 17 (depends on 11)

The image processing device according to claim 11,
configured such that an exposure time, sensitivity, a shooting gain, or a focusing position is different between the first and second shooting conditions.

### Claim 18

An image processing method comprising:
a first difference detection step of detecting difference information between a left-eye image and a right-eye image shot with a first shooting condition;
a second difference detection step of detecting difference information between the left-eye image and the right-eye image shot with a second shooting condition; and
a combining step of combining the difference information with the first shooting condition detected by a process at the first difference detection step and the difference information with the second shooting condition detected by a process at the second difference detection step on the basis of the left-eye image shot with the first or second shooting condition or the right-eye image shot with the first or second shooting condition and generating the combined difference information as a depth value
by an image processing device.

### Claim 19

A solid-state imaging device comprising:
a shooting unit which shoots a left-eye image and a right-eye image with first and second shooting conditions;
a first difference detection unit which detects difference information between the left-eye image and the right-eye image shot by the shooting unit with the first shooting condition;
a second difference detection unit which detects difference information between the left-eye image and the right-eye image shot by the shooting unit with the second shooting condition; and
a combining unit which combines the difference information with the first shooting condition detected by the first difference detection unit and the difference information with the second shooting condition detected by the second difference detection unit on the basis of the left-eye image shot with the first or second shooting condition or the right-eye image shot with the first or second shooting condition and generates the combined difference information as a depth value.
