# US 2015/0146096 A1 · Image-processing device and control method thereof

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2015/0146096 A1
- Document ID: US20150146096A1
- Status: Published application
- Kind: A1
- Application number: 14/541,294
- Filed: 2014-11-14
- Published: 2015-05-28
- Earliest claimed: 2013-11-27
- Assignees: Canon Kabushiki Kaisha
- Applicants: Canon Kabushiki Kaisha
- Inventors: Toshiki Nakatani; Izumi Kanai; Yohei Sekiguchi
- Web page: https://patentgrep.com/patent/US20150146096A1

## CPC

- H04N 5/144: {Movement detection (for video coding H04N19/503; analysis of motion in general G06T7/20)}
- H04N 5/145: {Movement estimation (for video coding H04N19/51)}
- H04N 7/0127: {by changing the field or frame frequency of the incoming video signal, e.g. frame rate converter}
- H04N 7/0132: {the field or frame frequency of the incoming video signal being multiplied by a positive integer, e.g. for flicker reduction}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

An image-processing device is configured to detect a motion region within an input image; generate an intermediate image from an N-th frame and an (N+1)-th frame; perform processing of reducing a high-frequency component of spatial frequency on the intermediate image; and output an image of which a frame rate is increased by insertion of the intermediate image having undergone the processing between the N-th frame and the (N+1)-th frame. When a target position at which a pixel value of the intermediate image is to be calculated lies within the motion region, a pixel value of the intermediate image at the target position is calculated using a pixel value of the N-th frame and a pixel value of the (N+1)-th frame at a position in the vicinity of the target position and outside the motion region.

## Claims

### Claim 1

An image-processing device, comprising:
a detection unit configured to detect a motion region in which there is motion between an N-th frame image and an (N+1)-th frame image of an input image;
a generation unit configured to generate an intermediate image from the N-th frame image and the (N+1)-th frame image, on the basis of a detection result by the detection unit;
a processing unit configured to perform on the intermediate image image processing of reducing a high-frequency component of spatial frequency; and
an output unit configured to output an image resulting from increasing a frame rate of the input image through insertion of the intermediate image having undergone the image processing, between the N-th frame image and the (N+1)-th frame image,
wherein when a target position at which a pixel value of the intermediate image is to be calculated lies within the motion region, the generation unit calculates a pixel value of the intermediate image at the target position, using a pixel value of the N-th frame image and a pixel value of the (N+1)-th frame image at a position in the vicinity of the target position and outside the motion region.

### Claim 2 (depends on 1)

The image-processing device according to claim 1, wherein the detection unit detects, as a motion region, a region at which the magnitude of an inter-frame difference, which is the difference between the pixel value of the N-th frame image and the pixel value of the (N+1)-th frame image, is equal to or greater than a predefined value.

### Claim 3 (depends on 1)

The image-processing device according to claim 1, wherein when the target position is within the motion region, the generation unit calculates a pixel value of the intermediate image at the target position on the basis of an average value of the pixel value of the N-th frame image and the pixel value of the (N+1)-th frame image at a position that is within a predefined range centered on the target position and outside the motion region.

### Claim 4 (depends on 1)

The image-processing device according to claim 1, wherein when the target position is within the motion region, the generation unit calculates a pixel value of the intermediate image at the target position on the basis of an average value of the pixel value of the N-th frame image and the pixel value of the (N+1)-th frame image at a position that is within a predefined range centered on the target position, outside the motion region, and that is closest to the target position.

### Claim 5 (depends on 3)

The image-processing device according to claim 3, wherein when there is no position that is within the predefined range centered on the target position and outside the motion region, the generation unit calculates a pixel value of the intermediate image at the target position on the basis of an average value of the pixel value of the N-th frame image and the pixel value of the (N+1)-th frame image at the target position.

### Claim 6 (depends on 3)

The image-processing device according to claim 3, wherein the predefined range comprises a range that is set, from the target position, in a direction of motion as detected by the detection unit, and a range that is set, from the target position, in a direction opposite the direction of motion as detected by the detection unit.

### Claim 7 (depends on 1)

The image-processing device according to claim 1, wherein when the target position is outside the motion region, the generation unit calculates a pixel value of the intermediate image at the target position on the basis of an average value of the pixel value of the N-th frame image and the pixel value of the (N+1)-th frame image at the target position.

### Claim 8 (depends on 3)

The image-processing device according to claim 3, wherein the generation unit generates a plurality of intermediate images including a first intermediate image and a second intermediate image,
the generation unit using, in order to calculate a pixel value of each pixel of the first intermediate image, pixel values of the N-th frame image and the (N+1)-th frame image at a first position,
the generation unit using, in order to calculate a pixel value of each pixel of the second intermediate image, pixel values of the N-th frame image and the (N+1)-th frame image at a second position,
the first position and the second position being different from each other.

### Claim 9 (depends on 8)

The image-processing device according to claim 8, wherein the generation unit sets the predefined range for each intermediate image in accordance with a time-axial position, between the N-th frame image and the (N+1)-th frame image, at which the intermediate image is inserted.

### Claim 10 (depends on 9)

The image-processing device according to claim 9, wherein the predefined range comprises a range that is set, from the target position, in a direction of motion as detected by the detection unit, and a range that is set, from the target position, in a direction opposite the direction of motion as detected by the detection unit, such that the closer the time-axial position of insertion of the intermediate image is to the N-th frame image, the wider the range is in the direction of the motion, and the closer the time-axial position of insertion of the intermediate image is to the (N+1)-th frame image, the wider the range is in the direction opposite the direction of the motion.

### Claim 11 (depends on 1)

The image-processing device according to claim 1, wherein the generation unit generates an intermediate image in which a pixel value varies continuously within the motion region, using a predefined interpolation function.

### Claim 12 (depends on 11)

The image-processing device according to claim 11, wherein the processing unit causes a degree at which the high-frequency component of spatial frequency is reduced to be different between the motion region and other regions, in the intermediate image.

### Claim 13 (depends on 1)

The image-processing device according to claim 1, wherein the generation unit generates an intermediate image in which a pixel value varies step-wise within the motion region.

### Claim 14 (depends on 13)

The image-processing device according to claim 13, wherein the generation unit
further generates a first average image and a second average image from the N-th frame image and the (N+1)-th frame image;
calculates a pixel value of the first average image at a target position, at which a pixel value of the first average image is to be calculated, on the basis of an average value of the pixel value of the N-th frame image at the target position and the pixel value of the (N+1)-th frame image at the target position;
calculates a pixel value of the second average image at a target position, at which a pixel value of the second average image is to be calculated, on the basis of an average value of pixel values of the first average image within a first range centered on the target position, when the target position is within the motion region;
sets, as a pixel value of the intermediate image at a target position, at which a pixel value of the intermediate image is to be calculated, a pixel value of the first average image at a position, from among positions within a second range centered on the target position, the magnitude of the difference between a pixel value of the first average image at the position and a pixel value of the second average image at the position being greatest, when the target position is within the motion region.

### Claim 15 (depends on 14)

The image-processing device according to claim 14, wherein the first range and the second range comprise a range that is set, from the target position, in a direction of motion as detected by the detection unit, and a range that is set, from the target position, in a direction opposite the direction of motion as detected by the detection unit;

### Claim 16 (depends on 14)

The image-processing device according to claim 14, wherein the second range is set in accordance with a speed of motion as detected by the detection unit.

### Claim 17 (depends on 16)

The image-processing device according to claim 16, wherein the second range is set to be wider the faster the speed of motion as detected by the detection unit is.

### Claim 18 (depends on 1)

The image-processing device according to claim 1, wherein
the processing unit further performs image processing of emphasizing the high-frequency component of spatial frequency on the N-th frame image and the (N+1)-th frame image; and
the output unit outputs an image resulting from increasing the frame rate of the input image through insertion of the intermediate image having undergone the image processing, between the N-th frame image and the (N+1)-th frame image having undergone the image processing.

### Claim 19

A control method of an image-processing device, the method comprising:
detecting a motion region in which there is motion between an N-th frame image and an (N+1)-th frame image of an input image;
generating an intermediate image from the N-th frame image and the (N+1)-th frame image, on the basis of a detection result in the detecting process;
performing on the intermediate image image processing of reducing a high-frequency component of spatial frequency; and
outputting an image resulting from increasing a frame rate of the input image through insertion of the intermediate image having undergone the image processing, between the N-th frame image and the (N+1)-th frame image,
wherein, in the generating process, when a target position at which a pixel value of the intermediate image is to be calculated lies within the motion region, a pixel value of the intermediate image at the target position is calculated using a pixel value of the N-th frame image and a pixel value of the (N+1)-th frame image at a position in the vicinity of the target position and outside the motion region.
