# US 2018/0324455 A1 · Image processing apparatus and method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0324455 A1
- Document ID: US20180324455A1
- Status: Published application
- Kind: A1
- Application number: 16/031,846
- Filed: 2018-07-10
- Published: 2018-11-08
- Earliest claimed: 2009-02-19
- Assignees: Sony Corporation
- Applicants: Sony Corporation
- Inventors: OHJI NAKAGAMI; YOICHI YAGASAKI
- Web page: https://patentgrep.com/patent/US20180324455A1

## CPC

- H04N 19/53: Multi-resolution motion estimation; Hierarchical motion estimation
- H04N 19/105: Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
- H04N 19/107: between spatial and temporal predictive coding, e.g. picture refresh
- H04N 19/109: among a plurality of temporal predictive coding modes
- H04N 19/117: Filters, e.g. for pre-processing or post-processing (sub-band filter banks H04N19/635)
- H04N 19/124: Quantisation
- H04N 19/33: in the spatial domain
- H04N 19/39: involving multiple description coding [MDC], i.e. with separate layers being structured as independently decodable descriptions of input picture data
- H04N 19/40: using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
- H04N 19/44: Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
- H04N 19/46: Embedding additional information in the video signal during the compression process (H04N19/517, H04N19/68, H04N19/70 take precedence)
- H04N 19/52: by predictive encoding
- H04N 19/573: Motion compensation with multiple frame prediction using two or more reference frames in a given prediction direction
- H04N 19/577: Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
- H04N 19/593: involving spatial prediction techniques
- H04N 19/61: in combination with predictive coding
- H04N 19/615: using motion compensated temporal filtering [MCTF]
- H04N 19/63: using sub-band based transform, e.g. wavelets
- H04N 19/80: Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation (H04N19/635, H04N19/86 take precedence)
- H04N 19/82: involving filtering within a prediction loop
- H04N 19/13: Adaptive entropy coding, e.g. adaptive variable length coding [AVLC] or context adaptive binary arithmetic coding [CABAC]
- H04N 19/513: Processing of motion vectors
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

The present invention relates to an image processing apparatus and method that can improve encoding efficiency while preventing an increase in load. An extraction circuit 71 of a filtering prediction circuit 64 extracts motion compensation images for generating a prediction image in a high-resolution enhancement layer from reference frames in a low-resolution base layer. A filtering circuit 72 of the filtering prediction circuit 64 performs filtering, which involves upconversion and which uses analysis in the time direction, on a plurality of motion compensation images in the base layer extracted by the extraction circuit 71 to generate a prediction image in the enhancement layer. The present invention can be applied to, for example, an encoding apparatus and a decoding apparatus.

## Claims

### Claim 1

An image decoding apparatus, comprising:
circuitry configured to:
increase a resolution of a first motion compensation image;
generate a difference image based on a difference between the first motion compensation image and a second motion compensation image;
filter, by a filtering process, at least one motion compensation image based on the difference image, wherein each of the first motion compensation image and the second motion compensation image is in a first layer; and
generate a prediction image in a second layer based on the first motion compensation image that has the increased resolution and the filtering process on the at least one motion compensation image.

### Claim 2 (depends on 1)

The image decoding apparatus according to claim 1, wherein a resolution of the first layer is different from a resolution of the second layer.

### Claim 3 (depends on 1)

The image decoding apparatus according to claim 1, wherein the circuitry is further configured to:
apply a low-pass filter to the difference image;
generate a first image based on the application of the low-pass filter;
apply a high-pass filter to the first image;
generate a second image based on the application of the high-pass filter; and
filter the at least one motion compensation image based on the first image and the second image.

### Claim 4 (depends on 1)

The image decoding apparatus according to claim 1, wherein the circuitry is further configured to decode an encoded image based on the generated prediction image.

### Claim 5 (depends on 1)

The image decoding apparatus according to claim 1, wherein the circuitry is further configured to increase a resolution of the difference image.

### Claim 6 (depends on 1)

The image decoding apparatus according to claim 1, wherein the circuitry is further configured to filter, by the filtering process, the at least one motion compensation image based on a correlation in a time direction that is included in the first motion compensation image and the second motion compensation image.

### Claim 7 (depends on 1)

The image decoding apparatus according to claim 1, wherein the circuitry is further configured to generate the prediction image by addition of the at least one motion compensation image subjected to the filtering process to the first motion compensation image that has the increased resolution.

### Claim 8

An image decoding method, comprising:
increasing a resolution of a first motion compensation image;
generating a difference image based on a difference between the first motion compensation image and a second motion compensation image;
filtering, by a filtering process, at least one motion compensation image based on the difference image, wherein each of the first motion compensation image and the second motion compensation image is in a first layer; and
generating a prediction image in a second layer based on the first motion compensation image that has the increased resolution and the filtering process on the at least one motion compensation image.

### Claim 9 (depends on 8)

The image decoding method according to claim 8, wherein a resolution of the first layer is different from a resolution of the second layer.

### Claim 10 (depends on 8)

The image decoding method according to claim 8, further comprising:
applying a low-pass filter to the difference image;
generating a first image based on the application of the low-pass filter;
applying a high-pass filter to the first image;
generating a second image based on the application of the high-pass filter; and
filtering the at least one motion compensation image based on the first image and the second image.

### Claim 11 (depends on 8)

The image decoding method according to claim 8, further comprising decoding an encoded image based on the generated prediction image.

### Claim 12 (depends on 8)

The image decoding method according to claim 8, further comprising increasing a resolution of the difference image.

### Claim 13 (depends on 8)

The image decoding method according to claim 8, further comprising filtering, by the filtering process, the at least one motion compensation image based on a correlation in a time direction that is included in the first motion compensation image and the second motion compensation image.

### Claim 14 (depends on 8)

The image decoding method according to claim 8, further comprising generating the prediction image by addition of the at least one motion compensation image subjected to the filtering process to the first motion compensation image that has the increased resolution.
