# US 2015/0085935 A1 · Sub-prediction unit (pu) based temporal motion vector prediction in hevc and sub-pu design in 3d-hevc

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2015/0085935 A1
- Document ID: US20150085935A1
- Status: Published application
- Kind: A1
- Application number: 14/496,991
- Filed: 2014-09-25
- Published: 2015-03-26
- Earliest claimed: 2013-09-26
- Assignees: QUALCOMM Incorporated
- Applicants: QUALCOMM Incorporated
- Inventors: Ying Chen; Geert Van der Auwera; Li Zhang; Hongbin Liu
- Web page: https://patentgrep.com/patent/US20150085935A1

## CPC

- H04N 19/597: specially adapted for multi-view video sequence encoding
- H04N 19/197: {including determination of the initial value of an encoding parameter (H04N19/56 takes precedence)}
- H04N 19/52: by predictive encoding
- H04N 19/53: Multi-resolution motion estimation; Hierarchical motion estimation
- H04N 19/56: Motion estimation with initialisation of the vector search, e.g. estimating a good candidate to initiate a search
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Techniques are described for sub-prediction unit (PU) based motion prediction for video coding in HEVC and 3D-HEVC. In one example, the techniques include an advanced temporal motion vector prediction (TMVP) mode to predict sub-PUs of a PU in single layer coding for which motion vector refinement may be allowed. The advanced TMVP mode includes determining motion vectors for the PU in at least two stages to derive motion information for the PU that includes different motion vectors and reference indices for each of the sub-PUs of the PU. In another example, the techniques include storing separate motion information derived for each sub-PU of a current PU predicted using a sub-PU backward view synthesis prediction (BVSP) mode even after motion compensation is performed. The additional motion information stored for the current PU may be used to predict subsequent PUs for which the current PU is a neighboring block.

## Claims

### Claim 1

A method of processing video data, the method comprising:
determining first motion information for a prediction unit (PU) of a coding unit (CU) of a video block from neighboring blocks of the PU according to a backward view synthesis prediction (BVSP) mode, wherein the first motion information includes at least one disparity motion vector and an associated reference view index that identifies an inter-view reference picture;
partitioning the PU into two or more sub-PUs;
determining second motion information for each of the sub-PUs, wherein the second motion information includes at least one disparity motion vector associated with a depth block of the inter-view reference picture corresponding to each of the sub-PUs;
performing motion compensation to predict each of the sub-PUs from the inter-view reference picture based on the second motion information; and
storing the second motion information for each of the sub-PUs of the PU in a memory to be used for predicting subsequent PUs.

### Claim 2 (depends on 1)

The method of claim 1, further comprising generating a merge mode candidate list of motion information for predicting a subsequent PU, wherein the PU is a neighboring block of the subsequent PU and generating the merge mode candidate list comprises including the second motion information for at least one of the sub-PUs of the PU stored in the memory as a sub-PU BVSP candidate in the merge mode candidate list.

### Claim 3 (depends on 1)

The method of claim 1, wherein determining the first motion information for the PU comprises selecting the first motion information as a BVSP candidate from a merge mode candidate list of motion information for predicting the PU.

### Claim 4 (depends on 1)

The method of claim 1, where determining the second motion information for each of the sub-PUs comprises deriving the at least one disparity motion vector for each of the sub-PUs to have a horizontal component equal to a maximum value of four corner pixels for the depth block of the inter-view reference picture corresponding to each of the sub-PUs, and a vertical component equal to zero.

### Claim 5 (depends on 1)

The method of claim 1, wherein determining the second motion information for each of the sub-PUs comprises selecting the second motion information as a sub-PU BVSP candidate from a merge mode candidate list of motion information for predicting sub-PUs.

### Claim 6 (depends on 1)

The method of claim 1, wherein the memory comprises a decoded picture buffer, and wherein storing the second motion information comprises storing the disparity motion vector for each of the sub-PUs in the decoded picture buffer with a reference picture list that includes the inter-view reference picture identified by the reference view index of the first motion information for the PU.

### Claim 7 (depends on 1)

The method of claim 1, further comprising:
generating a predictive block for each of the sub-PUs of the PU from the inter-view reference picture based on the second motion information;
generating a residual block based on the video block and the predictive block for each of the sub-PUs; and
encoding the residual block and an indicator of at least the first motion information for the PU in a video bitstream.

### Claim 8 (depends on 1)

The method of claim 1, further comprising:
decoding a residual block and an indicator of at least the first motion information for the PU from a received video bitstream;
generating a predictive block for each of the sub-PUs of the PU from the inter-view reference picture based on the second motion information; and
generating a reconstructed version of the video block based on the residual block and the predictive block for each of the sub-PUs.

### Claim 9 (depends on 1)

The method of claim 1, further comprising applying a deblocking filter to each CU of a reconstructed version of the video block prior to storing the video block in a decoded picture buffer, wherein applying the deblocking filter to the CU of the video block that includes the PU with the two or more sub-PUs comprises:
prior to applying the deblocking filter to the CU, creating artificial PU boundaries or artificial transform unit (TU) boundaries at sub-PU boundaries between two adjacent sub-PUs in the PU; and
applying the deblocking filter to at least one of PU boundaries between two adjacent PUs of the CU or TU boundaries between two adjacent TUs of the CU, including the artificial PU boundaries or the artificial TU boundaries.

### Claim 10 (depends on 9)

The method of claim 9, wherein creating the artificial PU boundaries or the artificial TU boundaries comprises converting a transform tree of the CU in order to associate the PU with a transform tree hierarchy and associate each of the sub-PUs with a TU, wherein the sub-PU boundaries comprise artificial TU boundaries.

### Claim 11 (depends on 9)

The method of claim 9, wherein creating the artificial PU boundaries or the artificial TU boundaries comprises converting a transform tree of the CU in order to associate the PU with a transform tree hierarchy and associate at least a part of each of the sub-PUs with a TU, wherein the sub-PU boundaries comprise artificial TU

### Claim 12 (depends on 9)

The method of claim 9, wherein creating the artificial PU boundaries or the artificial TU boundaries comprises converting the CU to a coding tree in order to associate the PU with a CU and associate each of the sub-PUs with a PU, wherein the sub-PU boundaries comprise artificial PU boundaries.

### Claim 13

A video processing device comprising:
a memory configured to store video data; and
one or more processors in communication with the memory and configured to:
determine first motion information for a prediction unit (PU) of a coding unit (CU) of a video block from neighboring blocks of the PU according to a backward view synthesis prediction (BVSP) mode, wherein the first motion information includes at least one disparity motion vector and an associated reference view index that identifies an inter-view reference picture,
partition the PU into two or more sub-PUs,
determine second motion information for each of the sub-PUs, wherein the second motion information includes at least one disparity motion vector associated with a depth block of the inter-view reference picture corresponding to each of the sub-PUs, and
perform motion compensation to predict each of the sub-PUs from the inter-view reference picture based on the second motion information,
wherein the memory is configured to store the second motion information for each of the sub-PUs of the PU to be used for predicting subsequent PUs.

### Claim 14 (depends on 13)

The device of claim 13, wherein the one or more processors are configured to generate a merge mode candidate list of motion information for predicting a subsequent PU, wherein the PU is a neighboring block of the subsequent PU and the merge mode candidate list includes the second motion information for at least one of the sub-PUs of the PU stored in the memory as a sub-PU BVSP candidate in the merge mode candidate list.

### Claim 15 (depends on 13)

The device of claim 13, wherein the one or more processors are configured to select the first motion information as a BVSP candidate from a merge mode candidate list of motion information for predicting the PU.

### Claim 16 (depends on 13)

The device of claim 13, where the one or more processors are configured to derive the at least one disparity motion vector included in the second motion information for each of the sub-PUs to have a horizontal component equal to a maximum value of four corner pixels for the depth block of the inter-view reference picture corresponding to each of the sub-PUs, and a vertical component equal to zero.

### Claim 17 (depends on 13)

The device of claim 13, wherein the one or more processors are configured to select the second motion information for each of the sub-PUs as a sub-PU BVSP candidate from a merge mode candidate list of motion information for predicting sub-PUs.

### Claim 18 (depends on 13)

The device of claim 13, wherein the memory comprises a decoded picture buffer, and wherein the one or more processors are configured to store the disparity motion vector included in the second motion information for each of the sub-PUs in the decoded picture buffer with a reference picture list that includes the inter-view reference picture identified by the reference view index of the first motion information for the PU.

### Claim 19 (depends on 13)

The device of claim 13, wherein the video processing device comprises a video encoding device, and wherein the one or more processors are configured to:
generate a predictive block for each of the sub-PUs of the PU from the inter-view reference picture based on the second motion information;
generate a residual block based on the video block and the predictive block for each of the sub-PUs; and
encode the residual block and an indicator of at least the first motion information for the PU in a video bitstream.

### Claim 20 (depends on 13)

The device of claim 13, wherein the video processing device comprises a video decoding device, and wherein the one or more processors are configured to:
decode a residual block and an indicator of at least the first motion information for the PU from a received video bitstream;
generate a predictive block for each of the sub-PUs of the PU from the inter-view reference picture based on the second motion information; and
generate a reconstructed version of the video block based on the residual block and the predictive block for each of the sub-PUs.

### Claim 21 (depends on 13)

The device of claim 13, wherein the one or more processors are configured to apply a deblocking filter to each CU of a reconstructed version of the video block prior to storing the video block in a decoded picture buffer, and wherein, to apply the deblocking filter to the CU of the video block that includes the PU with the two or more sub-PUs, the one or more processors are configured to:
prior to applying the deblocking filter to the CU, create artificial PU boundaries or artificial transform unit (TU) boundaries at sub-PU boundaries between two adjacent sub-PUs in the PU; and
apply the deblocking filter to at least one of PU boundaries between two adjacent PUs of the CU or TU boundaries between two adjacent TUs of the CU, including the artificial PU boundaries or the artificial TU boundaries.

### Claim 22 (depends on 21)

The device of claim 21, wherein the one or more processors are configured to convert a transform tree of the CU in order to associate the PU with a transform tree hierarchy and associate each of the sub-PUs with a TU, wherein the sub-PU boundaries comprise artificial TU boundaries.

### Claim 23 (depends on 21)

The device of claim 21, wherein the one or more processors are configured to convert a transform tree of the CU in order to associate the PU with a transform tree hierarchy and associate at least a part of each of the sub-PUs with a TU, wherein the sub-PU boundaries comprise artificial TU

### Claim 24 (depends on 21)

The device of claim 21, wherein the one or more processors are configured to convert the CU to a coding tree in order to associate the PU with a CU and associate each of the sub-PUs with a PU, wherein the sub-PU boundaries comprise artificial PU boundaries.

### Claim 25

A video processing device comprising:
means for determining first motion information for a prediction unit (PU) of a coding unit (CU) of a video block from neighboring blocks of the PU according to a backward view synthesis prediction (BVSP) mode, wherein the first motion information includes at least one disparity motion vector and an associated reference view index that identifies an inter-view reference picture;
means for partitioning the PU into two or more sub-PUs;
means for determining second motion information for each of the sub-PUs, wherein the second motion information includes at least one disparity motion vector associated with a depth block of the inter-view reference picture corresponding to each of the sub-PUs;
means for performing motion compensation to predict each of the sub-PUs from the inter-view reference picture based on the second motion information; and
means for storing the second motion information for each of the sub-PUs of the PU in a memory to be used for predicting subsequent PUs.

### Claim 26

A computer-readable storage medium storing instructions for processing video data that, when executed, cause one or more processors to:
determine first motion information for a prediction unit (PU) of a coding unit (CU) of a video block from neighboring blocks of the PU according to a backward view synthesis prediction (BVSP) mode, wherein the first motion information includes at least one disparity motion vector and an associated reference view index that identifies an inter-view reference picture;
partition the PU into two or more sub-PUs;
determine second motion information for each of the sub-PUs, wherein the second motion information includes at least one disparity motion vector associated with a depth block of the inter-view reference picture corresponding to each of the sub-PUs;
perform motion compensation to predict each of the sub-PUs from the inter-view reference picture based on the second motion information; and
store the second motion information for each of the sub-PUs of the PU in a memory to be used for predicting subsequent PUs.
