# US 2011/0254930 A1 · Image processing apparatus and image processing method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2011/0254930 A1
- Document ID: US20110254930A1
- Status: Published application
- Kind: A1
- Application number: 13/076,822
- Filed: 2011-03-31
- Published: 2011-10-20
- Earliest claimed: 2010-04-14
- Assignees: Canon Kabushiki Kaisha
- Inventors: Kousei Sugimoto
- Web page: https://patentgrep.com/patent/US20110254930A1

## CPC

- H04N 7/0127: {by changing the field or frame frequency of the incoming video signal, e.g. frame rate converter}
- H04N 13/111: Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
- H04N 5/145: {Movement estimation (for video coding H04N19/51)}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

An image processing apparatus includes: a unit that detects a motion vector from a first video image and a second video image; a unit that detects a disparity vector between a frame of the first video image and a frame of the second video image; a unit that, when there exists a non-detection position at which no motion vector is detected in a frame of one video image, performs correction by detecting a corresponding position included in a frame of the other video image that corresponds to the non-detection position using the disparity vector, and setting a motion vector of the frame of the other video image at the corresponding position, to the non-detection position; and a unit that generates respective interpolation frames of the first video image and the second video image using the motion vector after correction.

## Claims

### Claim 1

An image processing apparatus that generates interpolation frames between respective frames of a first video image and a second video image that constitute stereoscopic video, the image processing apparatus comprising:
a motion vector detection unit that detects a motion vector between frames of each video image using two temporally consecutive frames that constitute the first video image and two temporally consecutive frames that constitute the second video image;
a disparity vector detection unit that, using a frame that constitutes the first video image and a frame that constitutes the second video image, detects a disparity vector that represents disparity between the two frames;
a correction unit that, when there exists a non-detection position at which no motion vector is detected in a frame of one video image from among the first video image and the second video image, performs correction by detecting a corresponding position included in a frame of the other video image that corresponds to the non-detection position using the disparity vector, and setting a motion vector of the frame of the other video image at the corresponding position, to the non-detection position; and
an interpolation frame generation unit that generates respective interpolation frames of the first video image and the second video image using the motion vector after correction.

### Claim 2

An image processing apparatus that generates interpolation frames between respective frames of a first video image and a second video image that constitute stereoscopic video, the image processing apparatus comprising:
a motion vector detection unit that detects a motion vector between frames of each video image using two temporally consecutive frames that constitute the first video image and two temporally consecutive frames that constitute the second video image;
a disparity vector detection unit that, using a frame that constitutes the first video image and a frame that constitutes the second video image, detects a disparity vector that represents disparity between the two frames;
a correction unit that, when there exists a non-detection position at which no motion vector is detected in a frame of one video image from among the first video image and the second video image, performs correction by setting a zero-magnitude mot ion vector to the non-detection position, detecting a corresponding position included in a frame of the other video image that corresponds to the non-detection position using the disparity vector, and replacing a motion vector detected at the corresponding position by the zero-magnitude vector;
an interpolation frame generation unit that generates respective interpolation frames of the first video image and the second video image using the motion vector after correction.

### Claim 3 (depends on 1)

The image processing apparatus according to claim 1,
wherein the interpolation frame generation unit maps each pixel in a frame to a position in an interpolation frame, on a basis of a motion vector; generates thereafter each pixel of the interpolation frame using the pixels mapped to each position in the interpolation frame; and
generates pixels at the position of the interpolation frame to which a plurality of pixels are mapped, using only pixels at a position for which a motion vector detection result has been corrected by the correction unit, from among the plurality of pixels.

### Claim 4 (depends on 2)

The image processing apparatus according to claim 2,
wherein the interpolation frame generation unit maps each pixel in a frame to a position in an interpolation frame, on a basis of a motion vector; generates thereafter each pixel of the interpolation frame using the pixels mapped to each position in the interpolation frame; and
generates pixels at the position of the interpolation frame to which a plurality of pixels are mapped, using only pixels at a position for which a motion vector detection result has been corrected by the correction unit, from among the plurality of pixels.

### Claim 5

An image processing method, being a method executed by an image processing apparatus that generates interpolation frames between respective frames of a first video image and a second video image that constitute stereoscopic video, the method comprising the steps of:
detecting a motion vector between frames of each video image using two temporally consecutive frames that constitute the first video image and two temporally consecutive frames that constitute the second video image;
detecting, using a frame that constitutes the first video image and a frame that constitutes the second video image, a disparity vector that represents disparity between the two frames;
when there exists a non-detection position at which no motion vector is detected in a frame of one video image from among the first video image and the second video image, performing correction by detecting a corresponding position included in a frame of the other video image that corresponds to the non-detection position using the disparity vector, and setting a motion vector of the frame of the other video image at the corresponding position, to the non-detection position; and
generating respective interpolation frames of the first video image and the second video image using the motion vector after correction.

### Claim 6

An image processing method, being a method executed by an image processing apparatus that generates interpolation frames between respective frames of a first video image and a second video image that constitute stereoscopic video, the method comprising the steps of:
detecting a motion vector between frames of each video image using two temporally consecutive frames that constitute the first video image and two temporally consecutive frames that constitute the second video image;
detecting, using a frame that constitutes the first video image and a frame that constitutes the second video image, a disparity vector that represents disparity between the two frames;
when there exists a non-detection position at which no motion vector is detected in a frame of one video image from among the first video image and the second video image, performing correction by setting a zero-magnitude motion vector to the non-detection position, detecting a corresponding position included in a frame of the other video image that corresponds to the non-detection position using the disparity vector, and replacing a motion vector detected at the corresponding position by the zero-magnitude vector; and
generating respective interpolation frames of the first video image and the second video image using the motion vector after correction.
