# US 2010/0142619 A1 · Apparatus and method for processing image

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2010/0142619 A1
- Document ID: US20100142619A1
- Status: Published application
- Kind: A1
- Application number: 12/488,776
- Filed: 2009-06-22
- Published: 2010-06-10
- Earliest claimed: 2008-12-08
- Assignees: Kabushiki Kaisha Toshiba
- Inventors: Shingo Suzuki; Hirofumi Mori
- Web page: https://patentgrep.com/patent/US20100142619A1

## CPC

- H04N 19/587: involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
- H04N 19/107: between spatial and temporal predictive coding, e.g. picture refresh
- H04N 19/132: Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
- H04N 19/172: the region being a picture, frame or field
- H04N 19/44: Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
- H04N 19/51: Motion estimation or motion compensation
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

An asynchronous IDR detecting unit detects occurrence of scene change and, upon noticing that at the scene change superiority and inferiority of image quality can hardly be recognized by a human eye and a high-image-quality interpolation frame cannot be generated, a frame interpolation process controlling unit controls a frame interpolation unit to generate an interpolation frame by a simplified process before and after a case where a frame including the scene change (aperiodic IDP frame) is detected (necessity of frame interpolation is small) or generate a high-image-quality interpolation frame in the other case (a case where necessity of the frame interpolation is great).

## Claims

### Claim 1

An image processing apparatus comprising:
a receiving unit which receives data including a plurality of encoded moving image frames and encoding information referred to upon decoding each of the encoded moving image frames;
a decoder which decodes the data received by the receiving unit and extracts the plurality of moving image frames and the encoding information;
an interpolation frame generating unit which generates an interpolation frame to be inserted into the plurality of timely sequential moving image frames extracted by the decoder; and
a detector which detects a specific moving image frame in accordance with the encoding information extracted by the decoder,
wherein if the detector detects the specific moving image frame, the interpolation frame generating unit generates the interpolation frame inserted between the detected specific moving image frame and a moving image frame followed by the specific moving image frame by duplicating the detected specific moving image frame or the moving image frame followed by the specific moving image frame, and if the detector does not detect the specific moving image frame, the interpolation frame generating unit generates the interpolation frame inserted between a moving image frame which is not detected as the specific moving image frame and a moving image frame followed by the moving image frame in accordance with a motion vector obtained each of blocks constituting the interpolation frame.

### Claim 2 (depends on 1)

The apparatus according to claim 1, wherein if an aperiodic IDR frame is detected, the detector determines the detected frame as the specific moving image frame.

### Claim 3 (depends on 1)

The apparatus according to claim 1, wherein the detector measures an interval of the IDR frame and, if the measured interval is equal to or shorter than a predetermined time, the detector determines the IDR frame as the specific moving image frame.

### Claim 4 (depends on 1)

The apparatus according to claim 1, wherein if the specific moving image frame is not detected by the detector, the interpolation frame generating unit obtains the motion vector in accordance with a motion vector already obtained for a block which adjacent to the block for obtaining a motion vector in the interpolation frame and which has a motion vector obtained.

### Claim 5 (depends on 1)

The apparatus according to claim 1, wherein if the specific moving image frame is not detected by the detector, the interpolation frame generating unit determines the motion vector of the block constituting the interpolation frame as a motion vector of a block which constitutes the decoded moving image frame present before or after the interpolation frame and which is located at the same position as the block constituting the interpolation frame having the motion vector obtained.

### Claim 6

An image processing apparatus comprising:
a receiving unit which receives data including a plurality of encoded moving image frames and encoding information referred to upon decoding each of the encoded moving image frames;
a decoder which decodes the data received by the receiving unit and extracts the plurality of moving image frames and the encoding information;
an interpolation frame generating unit which generates an interpolation frame to be inserted into the plurality of timely sequential moving image frames extracted by the decoder; and
a detector which detects that a scene has been changed in accordance with the encoding information extracted by the decoder,
wherein if the detector detects that the scene has been changed, the interpolation frame generating unit generates the interpolation frame inserted between a moving image frame of the changed scene and a moving image frame followed by the moving image frame of the changed scene by duplicating the moving image frame or the moving image frame followed by the moving image frame of the changed scene, and if the detector does net detect that the scene has been changed, the interpolation frame generating unit generates the interpolation frame inserted between a moving image frame which is not detected that the scene has been changed and a moving image frame followed by the moving image frame in accordance with a motion vector obtained each of blocks constituting the interpolation frame.

### Claim 7 (depends on 6)

The apparatus according to claim 6, wherein if an aperiodic IDR frame is detected, the detector determines that the scene has been changed.

### Claim 8 (depends on 6)

The apparatus according to claim 6, wherein the detector measures an interval of the IDR frame and, if the measured interval is equal to or shorter than a predetermined time, the detector determines the scene has been changed.

### Claim 9 (depends on 6)

The apparatus according to claim 6, wherein if it is not detected by the detector that the scene has been changed, the interpolation frame generating unit obtains the motion vector in accordance with a motion vector already obtained for a block adjacent to a block which adjacent to the block for obtaining a motion vector constitutes in the interpolation frame and which has a motion vector obtained.

### Claim 10 (depends on 6)

The apparatus according to claim 6, wherein if it is not detected by the detector that the scene has been changed, the interpolation frame generating unit determines the motion vector of the block constituting the interpolation frame as a motion vector of a block which constitutes the decoded moving image frame present before or after the interpolation frame and which is located at the same position as the block constituting the interpolation frame having the motion vector obtained.

### Claim 11

An image processing method comprising:
receiving data including a plurality of encoded moving image frames and encoding information referred to upon decoding each of the encoded moving image frames;
decoding the data received by the receiving and extracting the plurality of moving image frames and the encoding information;
generating an interpolation frame to be inserted into the plurality of timely sequential moving image frames extracted by the decoding; and
detecting a specific moving image frame in accordance with the encoding information extracted by the decoding,
wherein if the specific moving image frame is detected by the detecting, the interpolation frame inserted between the detected specific moving image frame and a moving image frame followed by the specific moving image frame is generated by duplicating the detected specific moving image frame or the moving image frame followed by the specific moving image frame, and if the specific moving image frame is not detected, the interpolation frame inserted between a moving image frame which is not detected as the specific moving image frame and a moving image frame followed by the moving image frame is generated in accordance with a motion vector obtained each of blocks constituting the interpolation frame.

### Claim 12 (depends on 11)

The method according to claim 11, wherein if an aperiodic IDR frame is detected, the detecting determines the detected frame as the specific moving image frame.

### Claim 13 (depends on 11)

The method according to claim 11, wherein the detector measures an interval of the IDR frame and, if
the measured interval is equal to or shorter than a predetermined time, the detector determines the IDR frame as the specific moving image frame.

### Claim 14 (depends on 11)

The method according to claim 11, wherein if the specific moving image frame is not detected by the detecting, the interpolation frame generation obtains the motion vector in accordance with a motion vector already obtained for a block which adjacent to the block for obtaining a motion vector in the interpolation frame and which has a motion vector obtained.

### Claim 15 (depends on 11)

The method according to claim 11, wherein if the specific moving image frame is not detected by the detector, the interpolation frame generation determines the motion vector of the block constituting the interpolation frame as a motion vector of a block which constitutes the decoded moving image frame present before or after the interpolation frame and which is located at the same position as the block constituting the interpolation frame having the motion vector obtained.

### Claim 16

An image processing method comprising:
receiving data including a plurality of encoded moving image frames and encoding information referred to upon decoding each of the encoded moving image frames;
decoding the data received by the receiving and extracts the plurality of moving image frames and the encoding information;
generating an interpolation frame to be inserted into the plurality of timely sequential moving image frames extracted by the decoding; and
detecting that a scene has been changed in accordance with the encoding information extracted by the decoding,
wherein if it is detected by the detecting that the scene has been changed, the interpolation frame inserted between a moving image frame of the changed scene and a moving image frame followed by the moving image frame of the changed scene by duplicating the moving image frame or the moving image frame followed by the moving image frame of the changed scene, and if it is not detected that the scene has been changed, the interpolation frame inserted between a moving image frame of the scene whose change is not detected and a moving image frame followed by the moving image frame in accordance with a motion vector obtained each of blocks constituting the interpolation frame.

### Claim 17 (depends on 16)

The method according to claim 16, wherein if an aperiodic IDR frame is detected, the detecting determines that the scene has been changed.

### Claim 18 (depends on 16)

The apparatus according to claim 16, wherein the detector measures an interval of the IDR frame and, if the measured interval is equal to or shorter than a predetermined time, the detector determines the scene has been changed.

### Claim 19 (depends on 16)

The method according to claim 16, wherein if it is not detected by the detecting that the scene has been changed, the interpolation frame generation obtains the motion vector in accordance with a motion vector already obtained for a block adjacent to a block which adjacent to the block for obtaining a motion vector constitutes in the interpolation frame and which has a motion vector obtained.

### Claim 20 (depends on 16)

The method according to claim 16, wherein if it is not detected by the detecting that the scene has been changed, the interpolation frame generation determines the motion vector of the block constituting the interpolation frame as a motion vector of a block which constitutes the decoded moving image frame present before or after the interpolation frame and which is located at the same position as the block constituting the interpolation frame having the motion vector obtained.
