# US 12,120,353 B2 · Image decoding device, method, and program with prediction refinement optical flow

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,120,353 B2
- Document ID: US12120353B2
- Status: Granted
- Kind: B2
- Application number: 17/674,164
- Filed: 2022-02-17
- Published: 2022-06-02
- Granted: 2024-10-15
- Earliest claimed: 2019-09-20
- Assignees: Kddi Corporation
- Applicants: Kddi Corporation
- Inventors: Kyohei Unno; Kei Kawamura; Sei Naito
- Web page: https://patentgrep.com/patent/US12120353B2

## CPC

- H04N 19/70: characterised by syntax aspects related to video coding, e.g. related to compression standards
- H04N 19/139: Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability
- H04N 19/172: the region being a picture, frame or field
- H04N 19/174: the region being a slice, e.g. a line of blocks or a group of blocks
- H04N 19/46: Embedding additional information in the video signal during the compression process (H04N19/517, H04N19/68, H04N19/70 take precedence)
- H04N 19/50: using predictive coding (H04N19/61 takes precedence)
- H04N 19/513: Processing of motion vectors
- H04N 19/51: Motion estimation or motion compensation
- H04N 19/82: involving filtering within a prediction loop
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

An image decoding device includes: a decoding unit configured to decode a first flag which is included in a slice header and controls enabling/disabling of PROF processing from encoded data; and an affine prediction signal generation unit configured to perform the PROF processing in a case where the first flag indicates that the PROF processing is enabled in a slice corresponding to the slice header, and generate a prediction signal without performing the PROF processing in a case where the first flag indicates that the PROF processing is disabled in a slice corresponding to the slice header.

## Claims

### Claim 1

An image decoding device comprising a circuit, wherein the circuit:
decodes a first flag which is included in a sequence parameter set and controls enabling/disabling of PROF (Prediction Refinement with Optical Flow) processing from encoded data;
decodes a second flag indicating whether or not a flag indicating enabling/disabling of the PROF processing is included in a low order header of a picture parameter set from the sequence parameter set additionally in a case where the first flag indicates enabling of the PROF processing in units of sequences;
decodes a third flag which is included in a low order header of the picture parameter set and indicates enabling/disabling of the PROF processing from the encoded data in a case where the second flag indicates that the flag indicating enabling/disabling of the PROF processing is included in the low order header of a picture parameter set;
determines non-application conditions of the PROF processing for each block;
determines not to apply the PROF processing to a block that meets at least one of the non-application conditions of the PROF processing and generate a prediction signal without performing the PROF processing; and
calculates “diffMV” corresponding to each pixel position in each sub-block to which a block that doesn't meet any of the non-application conditions of the PROF processing is divided and performs correction of the prediction signal by executing the PROF processing with the “diffMV”,
the non-application conditions of the PROF processing include a condition that the first flag indicates disabling of the PROF processing in units of sequences,
the non-application conditions of the PROF processing include a condition that the third flag indicates disabling of the PROF processing, and
the non-application conditions of the PROF processing include a condition that “fallbackMode” is applied to a block.

### Claim 2

An image decoding method comprising:
decoding a first flag which is included in a sequence parameter set and controls enabling/disabling of PROF (Prediction Refinement with Optical Flow) processing from encoded data;
decoding a second flag indicating whether or not a flag indicating enabling/disabling of the PROF processing is included in a low order header of a picture parameter set from the sequence parameter set additionally in a case where the first flag indicates enabling of the PROF processing in units of sequences;
decoding a third flag which is included in a low order header of the picture parameter set and indicates enabling/disabling of the PROF processing from the encoded data in a case where the second flag indicates that the flag indicating enabling/disabling of the PROF processing is included in the low order header of a picture parameter set;
determining non-application conditions of the PROF processing for each block;
determining not to apply the PROF processing to a block that meets at least one of the non-application conditions of the PROF processing and generating a prediction signal without performing the PROF processing;
calculating “diffMV” corresponding to each pixel position in each sub-block to which a block that doesn't meet any of the non-application conditions of the PROF processing is divided, and performing correction of the prediction signal by executing the PROF processing with the “diffMV”,
the non-application conditions of the PROF processing include a condition that the first flag indicates disabling of the PROF processing in units of sequences,
the non-application conditions of the PROF processing include a condition that the third flag indicates disabling of the PROF processing, and
the non-application conditions of the PROF processing include a condition that “fallbackMode” is applied to a block.

### Claim 3

A program stored on a non-transitory computer-readable medium for causing a computer to execute:
decoding a first flag which is included in a sequence parameter set and controls enabling/disabling of PROF (Prediction Refinement with Optical Flow) processing from encoded data;
decoding a second flag indicating whether or not a flag indicating enabling/disabling of the PROF processing is included in a low order header of a picture parameter set from the sequence parameter set additionally in a case where the first flag indicates enabling of the PROF processing in units of sequences;
decoding a third flag which is included in a low order header of the picture parameter set and indicates enabling/disabling of the PROF processing from the encoded data in a case where the second flag indicates that the flag indicating enabling/disabling of the PROF processing is included in the low order header of a picture parameter set;
determining non-application conditions of the PROF processing for each block;
determining not to apply the PROF processing to a block that meets at least one of the non-application conditions of the PROF processing and generating a prediction signal without performing the PROF processing;
calculating “diffMV” corresponding to each pixel position in each sub-block to which a block that doesn't meet any of the non-application conditions of the PROF processing is divided, and performing correction of the prediction signal by executing the PROF processing with the “diffMV”,
the non-application conditions of the PROF processing include a condition that the first flag indicates disabling of the PROF processing in units of sequences,
the non-application conditions of the PROF processing include a condition that the third flag indicates disabling of the PROF processing, and
the non-application conditions of the PROF processing include a condition that “fallbackMode” is applied to a block.