# US 2020/0288175 A1 · Signaling of triangle merge mode indexes in video coding

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0288175 A1
- Document ID: US20200288175A1
- Status: Published application
- Kind: A1
- Application number: 16/809,774
- Filed: 2020-03-05
- Published: 2020-09-10
- Earliest claimed: 2019-03-06
- Assignees: QUALCOMM Incorporated
- Applicants: QUALCOMM Incorporated
- Inventors: Yao-Jen Chang; Chun-Chi Chen; Wei-Jung Chien; Marta Karczewicz
- Web page: https://patentgrep.com/patent/US20200288175A1

## CPC

- H04N 19/52: by predictive encoding
- H04N 19/70: characterised by syntax aspects related to video coding, e.g. related to compression standards
- H04N 19/119: Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
- H04N 19/137: Motion inside a coding unit, e.g. average field, frame or block difference
- H04N 19/176: the region being a block, e.g. a macroblock
- H04N 19/96: Tree coding, e.g. quad-tree coding
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A video decoder obtains a first triangle merging index syntax element specifying a first triangle merging candidate index. The first triangle merging candidate index indicates a first triangle merging candidate of a triangular shape-based motion compensation candidate list. The video decoder may determine whether the maximum number of triangle merging candidates is greater than 2. Based on the maximum number of triangle merging candidates not being greater than 2, the video decoder may infer that a second triangle merging candidate index indicates a second triangle merging candidate of the triangular shape-based motion compensation candidate list without obtaining any syntax element specifying the second triangle merging candidate index from the bitstream, the second triangle merging candidate being different from the first triangle merging candidate.

## Claims

### Claim 1

A method of decoding video data, the method comprising:
determining, based on a first syntax element signaled in a bitstream that includes an encoded representation of the video data, a maximum number of triangle merging candidates;
obtaining a first triangle merging index syntax element from the bitstream, the first triangle merging index syntax element specifying a first triangle merging candidate index, the first triangle merging candidate index indicating a first triangle merging candidate of a triangular shape-based motion compensation candidate list;
determining whether the maximum number of triangle merging candidates is greater than 2;
based on the maximum number of triangle merging candidates not being greater than 2, inferring that a second triangle merging candidate index indicates a second triangle merging candidate of the triangular shape-based motion compensation candidate list without obtaining any syntax element specifying the second triangle merging candidate index from the bitstream, the second triangle merging candidate being different from the first triangle merging candidate;
generating a prediction block for a coding unit (CU), wherein generating the prediction block for the CU comprises:
inter-predicting a first triangle partition of the CU using motion information of the first triangle merging candidate; and
inter-predicting a second triangle partition of the CU using motion information of the second triangle merging candidate; and
reconstructing the CU based on the prediction block for the CU and residual data for the CU.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
obtaining a third syntax element from the bitstream, the third syntax element indicating whether triangle merge mode is enabled; and
obtaining the first syntax element based on the third syntax element indicating that triangle merge mode is enabled.

### Claim 3 (depends on 1)

The method of claim 1, wherein:
the CU is a first CU,
the maximum number of triangle merging candidates is a first maximum number of triangle merging candidates,
the triangle shape-based motion compensation candidate list is a first triangle shape-based motion compensation candidate list, and
the method further comprises:
determining whether a second maximum number of triangle merging candidates is greater than 2;
obtaining a third triangle merging index syntax element from the bitstream, the third triangle merging index syntax element specifying a third triangle merging candidate index, the third triangle merging candidate index indicating a third triangle merging candidate of a second triangular shape-based motion compensation candidate list;
based on the second maximum number of triangle merging candidates being greater than 2, obtaining a fourth triangle merging index syntax element from the bitstream, the fourth triangle merging index syntax element specifying a fourth triangle merging candidate index, the fourth triangle merging candidate index indicating a fourth triangle merging candidate of the second triangle shape-based motion compensation candidate list;
generating a prediction block for a second coding unit (CU), wherein generating the prediction block for the second CU comprises:
inter-predicting a first triangle partition of the second CU using motion information of the third triangle merging candidate; and
inter-predicting a second triangle partition of the second CU using motion information of the fourth triangle merging candidate; and
reconstructing the second CU based on the prediction block for the second CU and residual data for the second CU.

### Claim 4 (depends on 1)

The method of claim 1, wherein determining the maximum number of triangle merging candidates comprises:
obtaining the first syntax element from the bitstream; and
determining the maximum number of triangle merging candidates as T minus a value specified by the first syntax element, where T is a preassigned positive integer.

### Claim 5 (depends on 4)

The method of claim 4, wherein T is equal to 5.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
generating the triangular shape-based motion compensation candidate list based on a table, wherein:
entries in the table correspond to different combinations of pairs of merging candidates in a merging candidate list,
the first and second triangle merging candidate indices specify respective entries in the table, and
the entries are ordered in the table so that no entry corresponding to a first pair of merging candidates is put behind another entry corresponding to a second pair of merging candidates if the second pair of merging candidates cannot be used given the maximum number of triangle merging candidates.

### Claim 7 (depends on 1)

The method of claim 1, further comprising:
generating the triangular shape-based motion compensation candidate list based on a selected table in a plurality of tables, wherein:
each table in the plurality of tables corresponds to a different maximum number of triangle merging candidates,
the selected table corresponds to the maximum number of triangle merging candidates that was determined based on the first syntax element, entries in the table correspond to different combinations of pairs of merging candidates in a merging candidate list, and
the first and second triangle merging candidate indices specify respective entries in the selected table.

### Claim 8 (depends on 1)

The method of claim 1, wherein the bitstream is encoded in accordance with Versatile Video Coding.

### Claim 9 (depends on 1)

The method of claim 1, wherein obtaining the first triangle merging index syntax element comprises obtaining the first triangle merging index syntax element from a merge data syntax structure.

### Claim 10

A method of encoding video data, the method comprising:
determining a maximum number of triangle merging candidates;
signaling a first triangle merging index syntax element in the bitstream, the first triangle merging index syntax element specifying a first triangle merging candidate index, the first triangle merging candidate index indicating a first triangle merging candidate of a triangular shape-based motion compensation candidate list;
determining whether the maximum number of triangle merging candidates is greater than 2;
based on the maximum number of triangle merging candidates not being greater than 2, omitting from the bitstream a second triangle merging index syntax element that specifies a second triangle merging candidate index indicating a second triangle merging candidate of the triangle shaped-based motion compensation candidate list, the second triangle merging candidate being different from the first triangle merging candidate;
generating a prediction block for a coding unit (CU), wherein generating the prediction block for the CU comprises:
inter-predicting a first triangle partition of the CU using motion information of the first triangle merging candidate; and
inter-predicting a second triangle partition of the CU using motion information of the second triangle merging candidate; and
generating residual data for the CU based on the prediction block for the CU and samples of the CU.

### Claim 11 (depends on 10)

The method of claim 10, further comprising:
signaling a second syntax element in the bitstream, the second syntax element indicating whether triangle merge mode is enabled; and
signaling a third syntax element based on the second syntax element indicating that triangle merge mode is enabled, wherein the third syntax element indicates the maximum number of triangle merging candidates.

### Claim 12 (depends on 10)

The method of claim 10, wherein:
the CU is a first CU,
the maximum number of triangle merging candidates is a first maximum number of triangle merging candidates,
the triangle shape-based motion compensation candidate list is a first triangle shape-based motion compensation candidate list, and
the method further comprises:
signaling a third triangle merging index syntax element in the bitstream, the third triangle merging index syntax element specifying a third triangle merging candidate index, the third triangle merging candidate index indicating a third triangle merging candidate of a second triangular shape-based motion compensation candidate list;
determining whether a second maximum number of triangle merging candidates is greater than 2;
based on the second maximum number of triangle merging candidates being greater than 2, signaling a fourth triangle merging index syntax element in the bitstream, the fourth triangle merging index syntax element specifying a fourth triangle merging candidate index, the fourth triangle merging candidate index indicating a fourth triangle merging candidate of the second triangle shape-based motion compensation candidate list;
generating a prediction block for a second coding unit (CU), wherein generating the prediction block for the second CU comprises:
inter-predicting a first triangle partition of the second CU using motion information of the third triangle merging candidate; and
inter-predicting a second triangle partition of the second CU using motion information of the fourth triangle merging candidate; and
reconstructing the second CU based on the prediction block for the second CU and residual data for the second CU.

### Claim 13 (depends on 10)

The method of claim 10, further comprising:
determining a value of a syntax element as T minus the maximum number of triangle merging candidates, where T is a preassigned positive integer; and
signaling the syntax element in the bitstream.

### Claim 14 (depends on 13)

The method of claim 13, wherein T is equal to 5.

### Claim 15 (depends on 10)

The method of claim 10, further comprising:
generating the triangular shape-based motion compensation candidate list based on a table, wherein:
entries in the table correspond to different combinations of pairs of merging candidates in a merging candidate list,
the first and second triangle merging candidate indices specify respective entries in the table, and
the entries are ordered in the table so that no entry corresponding to a first pair of merging candidates is put behind another entry corresponding to a second pair of merging candidates if the second pair of merging candidates cannot be used given the maximum number of triangle merging candidates.

### Claim 16 (depends on 10)

The method of claim 10, further comprising:
generating the triangular shape-based motion compensation candidate list based on a selected table in a plurality of tables, wherein:
each table in the plurality of tables corresponds to a different maximum number of triangle merging candidates,
the selected table corresponds to the maximum number of triangle merging candidates that was determined based on the first syntax element, entries in the table correspond to different combinations of pairs of merging candidates in a merging candidate list, and
the first and second triangle merging candidate indices specify respective entries in the table.

### Claim 17

A device for decoding video data, the device comprising:
a memory to store the video data; and
one or more processors implemented in circuitry, the one or more processors configured to:
determine, based on a first syntax element signaled in a bitstream that includes an encoded representation of the video data, a maximum number of triangle merging candidates;
obtain a first triangle merging index syntax element from the bitstream, the first triangle merging index specifying a first triangle merging candidate index, the first triangle merging candidate index indicating a first triangle merging candidate of a triangular shape-based motion compensation candidate list;
determine whether the maximum number of triangle merging candidates is greater than 2;
based on the maximum number of triangle merging candidates not being greater than 2, infer that a second triangle merging candidate index indicates a second triangle merging candidate of the triangular shape-based motion compensation candidate list without obtaining any syntax element specifying the second triangle merging candidate index from the bitstream, the second triangle merging candidate being different from the first triangle merging candidate;
generate a prediction block for a coding unit (CU), wherein the one or more processors are configured such that, as part of generating the prediction block for the CU, the one or more processors:
inter-predict a first triangle partition of the CU using motion information of the first triangle merging candidate; and
inter-predict a second triangle partition of the CU using motion information of the second triangle merging candidate; and
reconstruct the CU based on the prediction block for the CU and residual data for the CU.

### Claim 18 (depends on 17)

The device of claim 17, wherein the one or more processors are further configured to:
obtain a third syntax element from the bitstream, the third syntax element indicating whether triangle merge mode is enabled; and
obtain the first syntax element based on the third syntax element indicating that triangle merge mode is enabled.

### Claim 19 (depends on 17)

The device of claim 17, wherein:
the CU is a first CU,
the maximum number of triangle merging candidates is a first maximum number of triangle merging candidates,
the triangle shape-based motion compensation candidate list is a first triangle shape-based motion compensation candidate list, and
the one or more processors are further configured to:
determine whether a second maximum number of triangle merging candidates is greater than 2;
obtain a third triangle merging index syntax element from the bitstream, the third triangle merging index syntax element specifying a third triangle merging candidate index, the third merging candidate index indicating a third triangle merging candidate of a second triangular shape-based motion compensation candidate list;
based on the second maximum number of triangle merging candidates being greater than 2, obtain a fourth triangle merging index syntax element from the bitstream, the fourth triangle merging index syntax element specifying a fourth triangle merging candidate index, the fourth triangle merging candidate index indicating a fourth triangle merging candidate of the second triangle shape-based motion compensation candidate list;
generate a prediction block for a second coding unit (CU), wherein the one or more processors are configured such that, as part of generating the prediction block for the second CU, the one or more processors:
inter-predict a first triangle partition of the second CU using motion information of the third triangle merging candidate; and
inter-predict a second triangle partition of the second CU using motion information of the fourth triangle merging candidate; and
reconstruct the second CU based on the prediction block for the second CU and residual data for the second CU.

### Claim 20 (depends on 17)

The device of claim 17, wherein the one or more processors are configured such that, as part of determining the maximum number of triangle merging candidates, the one or more processors:
obtain the first syntax element from the bitstream; and
determine the maximum number of triangle merging candidates as T minus a value specified by the first syntax element, where T is a preassigned positive integer.

### Claim 21 (depends on 20)

The device of claim 20, wherein T is equal to 5.

### Claim 22 (depends on 17)

The device of claim 17, wherein the one or more processors are further configured to:
generate the triangular shape-based motion compensation candidate list based on a table, wherein:
entries in the table correspond to different combinations of pairs of merging candidates in a merging candidate list,
the first and second triangle merging candidate indices specify respective entries in the table, and
the entries are ordered in the table so that no entry corresponding to a first pair of merging candidates is put behind another entry corresponding to a second pair of merging candidates if the second pair of merging candidates cannot be used given the maximum number of triangle merging candidates.

### Claim 23 (depends on 17)

The device of claim 17, wherein the one or more processors are further configured to:
generate the triangular shape-based motion compensation candidate list based on a selected table in a plurality of tables, wherein:
each table in the plurality of tables corresponds to a different maximum number of triangle merging candidates,
the selected table corresponds to the maximum number of triangle merging candidates that was determined based on the first syntax element, entries in the table correspond to different combinations of pairs of merging candidates in a merging candidate list, and
the first and second triangle merging candidate indices specify respective entries in the table.

### Claim 24 (depends on 17)

The device of claim 17, further comprising a display configured to display decoded video data.

### Claim 25 (depends on 17)

The device of claim 17, wherein the device comprises one or more of a camera, a computer, a mobile device, a broadcast receiver device, or a set-top box.

### Claim 26

A device for encoding video data, the device comprising:
a memory to store the video data; and
one or more processors implemented in circuitry, the one or more processors configured to:
determine a maximum number of triangle merging candidates;
signal a first triangle merging index syntax element in the bitstream, the first triangle merging index syntax element specifying a first triangle merging candidate index, the first triangle merging candidate index indicating a first triangle merging candidate of a triangular shape-based motion compensation candidate list;
determine whether the maximum number of triangle merging candidates is greater than 2;
based on the maximum number of triangle merging candidates not being greater than 2, omit from the bitstream a second triangle merging index syntax element that specifies a second triangle merging candidate index indicating a second triangle merging candidate of the triangle shaped-based motion compensation candidate list, the second triangle merging candidate being different from the first triangle merging candidate;
generate a prediction block for a coding unit (CU), wherein the one or more processors are configured such that, as part of generating the prediction block for the CU, the one or more processors:
inter-predict a first triangle partition of the CU using motion information of the first triangle merging candidate; and
inter-predict a second triangle partition of the CU using motion information of the second triangle merging candidate; and
generate residual data for the CU based on the prediction block for the CU and samples of the CU.

### Claim 27 (depends on 26)

The device of claim 26, wherein the one or more processors are further configured to:
signal a second syntax element in the bitstream, the second syntax element indicating whether triangle merge mode is enabled; and
signal a third syntax element based on the second syntax element indicating that triangle merge mode is enabled, wherein the third syntax element indicates the maximum number of triangle merging candidates.

### Claim 28 (depends on 26)

The device of claim 26, wherein:
the CU is a first CU,
the maximum number of triangle merging candidates is a first maximum number of triangle merging candidates,
the triangle shape-based motion compensation candidate list is a first triangle shape-based motion compensation candidate list, and
the one or more processors are further configured to:
signal a third triangle merging index syntax element in the bitstream, the third triangle merging index syntax element specifying a third triangle merging candidate index, the third triangle merging candidate index indicating a third triangle merging candidate of a second triangular shape-based motion compensation candidate list;
determine whether a second maximum number of triangle merging candidates is greater than 2;
based on the second maximum number of triangle merging candidates being greater than 2, signal a fourth triangle merging index syntax element in the bitstream, the fourth triangle merging index syntax element specifying a fourth triangle merging candidate index, the fourth triangle merging candidate index indicating a fourth triangle merging candidate of the second triangle shape-based motion compensation candidate list;
generate a prediction block for a second coding unit (CU), wherein the one or more processors are configured such that, as part of generating the prediction block for the second CU, the one or more processors:
inter-predict a first triangle partition of the second CU using motion information of the third triangle merging candidate; and
inter-predict a second triangle partition of the second CU using motion information of the fourth triangle merging candidate; and
reconstruct the second CU based on the prediction block for the second CU and residual data for the second CU.

### Claim 29 (depends on 26)

The device of claim 26, wherein the one or more processors are further configured to:
determine a value of a syntax element as T minus the maximum number of triangle merging candidates, where T is a preassigned positive integer; and
signal the syntax element in the bitstream.

### Claim 30 (depends on 29)

The device of claim 29, wherein T is equal to 5.

### Claim 31 (depends on 26)

The device of claim 26, wherein the one or more processors are further configured to:
generate the triangular shape-based motion compensation candidate list based on a table, wherein:
entries in the table correspond to different combinations of pairs of merging candidates in a merging candidate list,
the first and second triangle merging candidate indices specify respective entries in the table, and
the entries are ordered in the table so that no entry corresponding to a first pair of merging candidates is put behind another entry corresponding to a second pair of merging candidates if the second pair of merging candidates cannot be used given the maximum number of triangle merging candidates.

### Claim 32 (depends on 26)

The device of claim 26, wherein the one or more processors are further configured to:
generate the triangular shape-based motion compensation candidate list based on a selected table in a plurality of tables, wherein:
each table in the plurality of tables corresponds to a different maximum number of triangle merging candidates,
the selected table corresponds to the maximum number of triangle merging candidates that was determined based on the first syntax element,
entries in the table correspond to different combinations of pairs of merging candidates in a merging candidate list, and
the first and second triangle merging candidate indices specify respective entries in the table.

### Claim 33 (depends on 26)

The device of claim 26, wherein the device comprises one or more of a camera, a computer, a mobile device, a broadcast receiver device, or a set-top box.

### Claim 34

A device for decoding video data, the device comprising:
means for determining, based on a first syntax element signaled in a bitstream that includes an encoded representation of the video data, a maximum number of triangle merging candidates;
means for obtaining a first triangle merging index syntax element from the bitstream, the first triangle merging index syntax element specifying a first triangle merging candidate index, the first triangle merging candidate index indicating a first triangle merging candidate of a triangular shape-based motion compensation candidate list;
means for determining whether the maximum number of triangle merging candidates is greater than 2;
means for inferring, based on the maximum number of triangle merging candidates not being greater than 2, that a second triangle merging candidate index indicates a second triangle merging candidate of the triangular shape-based motion compensation candidate list without obtaining any syntax element specifying the second triangle merging candidate index from the bitstream, the second triangle merging candidate being different from the first triangle merging candidate;
means for generating a prediction block for a coding unit (CU), wherein the means for generating the prediction block for the CU comprises:
means for inter-predicting a first triangle partition of the CU using motion information of the first triangle merging candidate; and
means for inter-predicting a second triangle partition of the CU using motion information of the second triangle merging candidate; and
means for reconstructing the CU based on the prediction block for the CU and residual data for the CU.

### Claim 35

A device for encoding video data, the device comprising:
means for determining a maximum number of triangle merging candidates;
means for signaling a first triangle merging index syntax element in the bitstream, the first triangle merging index syntax element specifying a first triangle merging candidate index, the first triangle merging candidate index indicating a first triangle merging candidate of a triangular shape-based motion compensation candidate list;
means for determining whether the maximum number of triangle merging candidates is greater than 2;
means for omitting, based on the maximum number of triangle merging candidates not being greater than 2, from the bitstream a second triangle merging index syntax element that specifies a second triangle merging candidate index indicating a second triangle merging candidate of the triangle shaped-based motion compensation candidate list, the second triangle merging candidate being different from the first triangle merging candidate;
means for generating a prediction block for a coding unit (CU), wherein the means for generating the prediction block for the CU comprises:
means for inter-predicting a first triangle partition of the CU using motion information of the first triangle merging candidate; and
means for inter-predicting a second triangle partition of the CU using motion information of the second triangle merging candidate; and
means for generating residual data for the CU based on the prediction block for the CU and samples of the CU.

### Claim 36

A computer-readable data storage medium having instructions stored thereon that, when executed, cause one or more processors to:
determine, based on a first syntax element signaled in a bitstream that includes an encoded representation of video data, a maximum number of triangle merging candidates;
obtain a first triangle merging index syntax element from the bitstream, the first triangle merging index syntax element specifying a first triangle merging candidate index, the first triangle merging candidate index indicating a first triangle merging candidate of a triangular shape-based motion compensation candidate list;
determine whether the maximum number of triangle merging candidates is greater than 2;
based on the maximum number of triangle merging candidates not being greater than 2, infer that a second triangle merging candidate index indicates a second triangle merging candidate of the triangular shape-based motion compensation candidate list without obtaining any syntax element specifying the second triangle merging candidate index from the bitstream, the second triangle merging candidate being different from the first triangle merging candidate;
generate a prediction block for a coding unit (CU), wherein generating the prediction block for the CU comprises:
inter-predicting a first triangle partition of the CU using motion information of the first triangle merging candidate; and
inter-predicting a second triangle partition of the CU using motion information of the second triangle merging candidate; and
reconstruct the CU based on the prediction block for the CU and residual data for the CU.

### Claim 37

A computer-readable data storage medium having instructions stored thereon that, when executed, cause one or more processors to:
determine a maximum number of triangle merging candidates;
signal a first triangle merging index syntax element in the bitstream, the first triangle merging index specifying a first triangle merging candidate index, the first triangle merging candidate index indicating a first triangle merging candidate of a triangular shape-based motion compensation candidate list;
determine whether the maximum number of triangle merging candidates is greater than 2;
based on the maximum number of triangle merging candidates not being greater than 2, omit from the bitstream a second triangle merging index syntax element that specifies a second triangle merging candidate index indicating a second triangle merging candidate of the triangle shaped-based motion compensation candidate list, the second triangle merging candidate being different from the first triangle merging candidate;
generate a prediction block for a coding unit (CU), wherein generating the prediction block for the CU comprises:
inter-predicting a first triangle partition of the CU using motion information of the first triangle merging candidate; and
inter-predicting a second triangle partition of the CU using motion information of the second triangle merging candidate; and
generate residual data for the CU based on the prediction block for the CU and samples of the CU.