# US 2020/0288128 A1 · Encoding apparatus and encoding method as well as decoding apparatus and decoding method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0288128 A1
- Document ID: US20200288128A1
- Status: Published application
- Kind: A1
- Application number: 16/880,650
- Filed: 2020-05-21
- Published: 2020-09-10
- Earliest claimed: 2016-04-22
- Assignees: Sony Corporation
- Applicants: Sony Corporation
- Inventors: KEISUKE CHIDA; TAKAHIRO NAGANO; KENICHIRO HOSOKAWA; TAKURO KAWAI; OHJI NAKAGAMI
- Web page: https://patentgrep.com/patent/US20200288128A1

## CPC

- H04N 19/117: Filters, e.g. for pre-processing or post-processing (sub-band filter banks H04N19/635)
- G06T 7/40: Analysis of texture (depth or shape recovery from texture G06T7/529)
- H04N 19/11: among a plurality of spatial predictive coding modes
- H04N 19/14: Coding unit complexity, e.g. amount of activity or edge presence estimation (H04N19/146 takes precedence)
- H04N 19/147: according to rate distortion criteria (rate-distortion as a criterion for motion estimation H04N19/567)
- H04N 19/172: the region being a picture, frame or field
- H04N 19/176: the region being a block, e.g. a macroblock
- H04N 19/65: using error resilience
- G06T 2207/10016: Video; Image sequence
- G06T 2207/20024: Filtering details
- H04N 19/94: Vector quantisation
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

The encoding apparatus encodes an input image by a non-reversible encoding method and transmits identification information for identifying match components that are, from among a plurality of texture components registered in a database, texture components that match with the input image and encoded data obtained by encoding the input image. The decoding apparatus receives the encoded data and the identification information, decodes the encoded data into a decoded image, and synthesizes the texture component as the match component identified by the identification information from among a plurality of texture components registered in a database and the decoded image. The present technology can be applied, for example, a codec or the like by which an image is encoded and decoded.

## Claims

### Claim 1

An encoding apparatus, comprising:
an encoding unit configured to encode an input image by a non-reversible encoding method;
a database configured to register a plurality of texture components;
a separation unit configured to separate a low frequency component of the input image from the input image;
a basis synthesis unit configured to generate a restoration component for each texture component of the plurality of texture components registered in the database, wherein
the restoration component restores a texture component of the input image, and
the restoration component is generated based on a basis synthesis in which the low frequency component of the input image and a basis of a texture component of the plurality of texture components are used;
a match component determination unit configured to determine, as a match component, the restoration component whose error with respect to the input image is a minimum from among a plurality of restoration components generated for the plurality of texture components registered in the database; and
a transmission unit configured to transmit identification information and encoded data, wherein
the identification information is for identification of the match component from among the plurality of texture components registered in the database, the match component is a texture component that matches with the input image, and
the encoded data is obtained by encode the input image.

### Claim 2

An encoding apparatus, comprising:
an encoding unit configured to encode an input image by a non-reversible encoding method;
a database configured to:
register a plurality of texture components; and
register a plurality of bases of the plurality of texture components, wherein the plurality of bases is obtained based on conversion of each texture component of the plurality of texture components into a basis;
a registration unit configured to:
register a basis of the texture component into the database;
temporarily register the basis of the texture component of the plurality of texture components of the input image as a basis of a new texture component into the database; and
definitively register the basis of the new texture component based on satisfaction of a given condition; and
a transmission unit configured to transmit identification information and encoded data, wherein
the identification information is for identification of the match component from among the plurality of texture components registered in the database,
the match component is a texture component that matches with the input image, and
the encoded data is obtained by encode the input image.

### Claim 3 (depends on 2)

The encoding apparatus according to claim 2, wherein the given condition is one of:
an S/N (Signal to Noise ratio) of the match component where the basis of the new texture component is registered in the database with respect to the input image is superior by a first fixed value or more to an S/N of the match component where the basis of the new texture component is not registered in the database with respect to the input image; or
an RD (Rate-Distortion) curve where the basis of the new texture component is registered in the database is superior by a second fixed value or more to an RD curve where the basis of the new texture component is not registered in the database.

### Claim 4 (depends on 2)

The encoding apparatus according to claim 2, wherein, based on an error of the match component where the basis of the new texture component is not registered in the database with respect to the input image that is equal to or greater than a threshold value, the registration unit is further configured to temporarily register the basis of the texture component of the plurality of texture components of the input image as the basis of the new texture component into the database.

### Claim 5

An encoding method, comprising:
encoding an input image by a non-reversible encoding method;
registering a plurality of texture components in a database;
separating a low frequency component of the input image from the input image;
generating a restoration component for each texture component of the plurality of texture components registered in the database, wherein
the restoration component restores a texture component of the input image, and
the restoration component is generated based on a basis synthesis in which the low frequency component of the input image and a basis of a texture component of the plurality of texture components are used;
determining, as a match component, the restoration component whose error with respect to the input image is a minimum from among a plurality of restoration components generated for the plurality of texture components registered in the database; and
transmitting identification information and encoded data, wherein
the identification information is for identification of the match component from among the plurality of texture components registered in the database,
the match component is a texture component that matches with the input image, and
the encoded data is obtained by encoding the input image.

### Claim 6

An encoding method, comprising:
encoding an input image by a non-reversible encoding method;
registering a plurality of texture components; and
registering a plurality of bases of the plurality of texture components, wherein the plurality of bases is obtained based on conversion of each texture component of the plurality of texture components into a basis;
registering a basis of the texture component into the database;
temporarily registering the basis of the texture component of the plurality of texture components of the input image as a basis of a new texture component into the database; and
definitively registering the basis of the new texture component based on satisfaction of a given condition; and
transmitting identification information and encoded data, wherein
the identification information is for identification of the match component from among the plurality of texture components registered in the database,
the match component is a texture component that matches with the input image, and
the encoded data is obtained by encoding the input image.

### Claim 7

A decoding apparatus, comprising:
a reception unit configured to receive encoded data and identification information from an encoding apparatus, wherein
the encoded data is obtained by encode of an input image by a non-reversible encoding method,
the identification information is for identification of a match component that is a texture component that matches with the input image,
the match component corresponds to a restoration component whose error with respect to the input image is a minimum from among a plurality of restoration components generated for a plurality of texture components,
the plurality of restoration components is generated by the encoding apparatus based on a first basis synthesis in which a first decoded image and a basis of a texture component of the plurality of texture components are used,
the encoded data is decoded into the first decoded image by the encoding apparatus, and
the restoration component restores a texture component of the input image;
a decoding unit configured to decode the encoded data into a second decoded image;
a database configured to:
register the plurality of texture components; and
register a plurality of bases of the plurality of texture components, wherein the plurality of bases is obtained based on conversion of each texture component of the plurality of texture components into a basis;
a basis synthesis unit configured to generate a restored component based on a second basis synthesis in which the second decoded image or a low frequency component of the second decoded image and a basis of the match component are used; and
a synthesis unit configured to synthesize the restored component, the second decoded image and the match component, wherein
the match component is identified from among the plurality of texture components registered in the database, and
the match component is identified by the identification information.

### Claim 8 (depends on 7)

The decoding apparatus according to claim 7, further comprising: an updating unit configured to request the encoding apparatus, which encodes the input image, and acquire data to be registered into the database to update the database.

### Claim 9 (depends on 7)

The decoding apparatus according to claim 7, further comprising an updating unit configured to acquire data from a server to update the database.

### Claim 10

A decoding method, comprising:
receiving encoded data and identification information from an encoding apparatus, wherein
the encoded data is obtained by encoding an input image by a non-reversible encoding method,
the identification information is for identification of a match component that is a texture component that matches with the input image,
the match component corresponds to a restoration component whose error with respect to the input image is a minimum from among a plurality of restoration components generated for a plurality of texture components, and
the plurality of restoration components is generated by the encoding apparatus based on a first basis synthesis in which a first decoded image and a basis of a texture component of the plurality of texture components are used,
the encoded data is decoded into the first decoded image by the encoding apparatus;
restoring a texture component of the input image;
decoding the encoded data into a second decoded image;
registering the plurality of texture components; and
registering a plurality of bases of the plurality of texture components, wherein the plurality of bases is obtained based on conversion of each texture component of the plurality of texture components into a basis;
generating a restored component based on a second basis synthesis in which the second decoded image or a low frequency component of the second decoded image and a basis of the match component are used; and
synthesizing the restored component, the second decoded image and the match component, wherein
the match component is identified from among the plurality of texture components registered in the database, and
the match component is identified by the identification information.