# US 10,375,415 B2 · Image processing apparatus and method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,375,415 B2
- Document ID: US10375415B2
- Status: Granted
- Kind: B2
- Application number: 15/934,129
- Filed: 2018-03-23
- Published: 2018-07-26
- Granted: 2019-08-06
- Earliest claimed: 2012-12-21
- Assignees: Sony Corporation
- Applicants: Sony Corporation
- Inventors: Yoshitomo Takahashi; Ohji Nakagami
- Web page: https://patentgrep.com/patent/US10375415B2

## CPC

- H04N 19/597: specially adapted for multi-view video sequence encoding
- H04N 19/521: {for estimating the reliability of the determined motion vectors or motion vector field, e.g. for smoothing the motion vector field or for correcting motion vectors}
- H04N 19/573: Motion compensation with multiple frame prediction using two or more reference frames in a given prediction direction
- H04N 19/58: Motion compensation with long-term prediction, i.e. the reference frame for a current frame not being the temporally closest one (H04N19/23 takes precedence)
- H04N 19/70: characterised by syntax aspects related to video coding, e.g. related to compression standards
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

The present disclosure relates to an image processing apparatus and a method that enable efficient change of reference relationship of views in multi-view coding. In the present technology, as for inter-view reference images, a used_by_curr flag is also set to an image having a possibility of being referred, in a slice header or syntax of an SPS. Only an image with the used_by_curr flag of 1 is added to a temporal list used in generating a reference list. Therefore, the temporal list (RefPicListTemp0[rIdx]) of L0 is generated in an order of a short-term (before) reference image of an index 0, a short-term (after) reference image of an index 1, and inter-view reference images of indexes 2 and 3. The present disclosure can be applied to an image processing apparatus, for example.

## Claims

### Claim 1

An image processing apparatus, comprising:
circuitry configured to:
decode a bit stream, and generate a first image based on the bit stream;
generate a default reference list based on identification information that identifies layer reference images of layers, which are set in a video parameter set (VPS), and which are referable,
wherein the layer reference images are added to the default reference list to change a reference relationship of a plurality of views for the first image,
the identification information is set in at least one of a sequence parameter set (SPS) or a slice header for each view of the plurality of views, and
the slice header is at a lower level relative to the VPS;
reverse an order of the layer reference images with respect to an L0 direction;
insert the layer reference images into the default reference list, with respect to an L1 direction; and
predict the first image by reference to the default reference list.

### Claim 2 (depends on 1)

The image processing apparatus according to claim 1, wherein the circuitry is further configured to:
add one layer reference image of the layer reference images, in which the identification information indicates 1; and
generate the default reference list based on the added one layer reference image.

### Claim 3 (depends on 1)

The image processing apparatus according to claim 1, wherein the circuitry is further configured to:
add one layer reference image of the layer reference images, in which the identification information indicates 1;
combine reference images in an order of a first reference image temporally before the first image, a second reference image temporally after the first image, a long-term reference image, and the layer reference images with respect to the L0 direction; and
generate the default reference list based on the combined reference images.

### Claim 4 (depends on 1)

The image processing apparatus according to claim 1, wherein the circuitry is further configured to:
add one layer reference image of the layer reference images, in which the identification information indicates 1; and
combine reference images in an order of a first reference image temporally after the first image, a second reference image temporally before the first image, a long-term reference image, and the layer reference images with respect to the L1 direction; and
generate the default reference list based on the combined reference images.

### Claim 5 (depends on 1)

The image processing apparatus according to claim 1, wherein the layers correspond to the plurality of views.

### Claim 6

An image processing method, comprising:
in an image processing apparatus:
decoding a bit stream;
generating a first image based on the bit stream;
generating a default reference list based on identification information that identifies layer reference images of layers, which are set in a video parameter set (VPS), and which are referable,
wherein the layer reference images are added to the default reference list to change a reference relationship of a plurality of views for the first image,
the identification information is set in at least one of a sequence parameter set (SPS) or a slice header for each view of the plurality of views, and
the slice header is at a lower level relative to the VPS;
reversing an order of the layer reference images with respect to an L0 direction;
inserting the layer reference images into the default reference list, with respect to an L1 direction; and
predicting the first image by reference to the default reference list.

### Claim 7

An image processing apparatus, comprising:
circuitry configured to:
set layers referable in a video parameter set (VPS);
set identification information in at least one of a sequence parameter set (SPS) or a slice header for each view of a plurality of views, wherein the slice header is at a lower level layer relative to the VPS;
generate a default reference list based on the identification information, wherein the identification information identifies layer reference images added to the default reference list to change a reference relationship of the plurality of views for a first image;
reverse an order of the layer reference images with respect to an L0 direction;
insert the layer reference images into the default reference list, with respect to an L1 direction;
generate a predicted image by reference to the default reference list;
encode the first image based on the predicted image;
generate a bit stream based on the first image; and
transmit the bit stream and the identification information.

### Claim 8 (depends on 7)

The image processing apparatus according to claim 7, wherein the circuitry is further configured to:
add one layer reference image of the layer reference images, in which the identification information indicates 1; and
generate the default reference list based on the added one layer reference image.

### Claim 9 (depends on 7)

The image processing apparatus according to claim 7, wherein the circuitry is further configured to:
add one layer reference image of the layer reference images, in which the identification information indicates 1;
combine reference images in an order of a first reference image temporally before the first image, a second reference image temporally after the first image, a long-term reference image, and the layer reference images with respect to the L0 direction; and
generate the default reference list based on the combined reference images.

### Claim 10 (depends on 7)

The image processing apparatus according to claim 7, wherein the circuitry is further configured to:
add one layer reference image of the layer reference images referable in a layer direction, in which the identification information indicates 1;
combine reference images in an order of a first reference image temporally after the first image, a second reference image temporally before the first image, a long-term reference image, and the layer reference images with respect to the L1 direction; and
generate the default reference list based on the combined reference images.

### Claim 11 (depends on 7)

The image processing apparatus according to claim 7, wherein the layers correspond to the plurality of views.

### Claim 12

An image processing method, comprising:
in an image processing apparatus:
setting layers referable in a video parameter set (VPS);
setting identification information in at least one of a sequence parameter set (SPS) or a slice header for each view of a plurality of views, wherein the slice header is at a lower level layer relative to the VPS;
generating a default reference list based on the identification information, wherein the identification information identifies layer reference images added to the default reference list to change a reference relationship of the plurality of views for a first image;
reversing an order of the layer reference images with respect to an L0 direction;
inserting the layer reference images into the default reference list, with respect to an L1 direction;
generating a predicted image by reference to the default reference list;
encoding the first image based on the predicted image;
generating a bit stream based on the first image; and
transmitting the bit stream and the identification information.