# US 12,003,758 B2 · Signaling of in-loop reshaping information using parameter sets

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,003,758 B2
- Document ID: US12003758B2
- Status: Granted
- Kind: B2
- Application number: 17/949,045
- Filed: 2022-09-20
- Published: 2023-02-16
- Granted: 2024-06-04
- Earliest claimed: 2019-02-01
- Assignees: Beijing Bytedance Network Technology Co., Ltd.; Bytedance Inc.
- Applicants: Beijing Bytedance Network Technology Co., Ltd.; Bytedance Inc.
- Inventors: Li Zhang; Kai Zhang; Hongbin Liu; Jizheng Xu; Yue Wang
- Web page: https://patentgrep.com/patent/US12003758B2

## CPC

- H04N 19/52: by predictive encoding
- H04N 19/105: Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
- H04N 19/117: Filters, e.g. for pre-processing or post-processing (sub-band filter banks H04N19/635)
- H04N 19/132: Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
- H04N 19/159: Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
- H04N 19/176: the region being a block, e.g. a macroblock
- H04N 19/184: the unit being bits, e.g. of the compressed video stream
- H04N 19/186: the unit being a colour or a chrominance component
- H04N 19/30: using hierarchical techniques, e.g. scalability (H04N19/63 takes precedence)
- H04N 19/70: characterised by syntax aspects related to video coding, e.g. related to compression standards
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A method for video processing includes performing a conversion between a current video block of a video region of a video and a coded representation of the video. The conversion uses a coding mode in which the current video block is constructed based on a first domain and a second domain and/or chroma residue is scaled in a luma-dependent manner. A parameter set in the coded representation comprises parameter information for the coding mode.

## Claims

### Claim 1

A method of processing video data, comprising:
performing a conversion between a current video block of a video region of a video and a bitstream of the video,
wherein the conversion uses a coding tool in which the current video block is constructed based on at least one of:
1) A forward mapping process for a luma component of the current video block, in which samples of the luma component are converted from an original domain to a reshaped domain,
2) An inverse mapping process, which is an inverse operation of the forward mapping process to convert the samples of the luma component in the reshaped domain to the original domain, or
3) A scaling process, in which residual samples of a chroma component of the current video block are scaled before being used to reconstruct the chroma component of the current video block,
wherein an adaptation parameter set is included in the bitstream which comprises parameter information for the coding tool and a first identifier of the adaptation parameter set,
wherein the first identifier of the adaptation parameter set is fixed-length coded and has a range between 0 to M, wherein M is 2K−1, and
wherein the parameter information includes at least one of an indication of information of a reshaper model which is used in the forward mapping process to map the samples of the luma component into particular values or chroma residual scaling factors which are used in the scaling process to derive scaled residual samples of the chroma component.

### Claim 2 (depends on 1)

The method of claim 1, wherein the adaptation parameter set is allowed to be shared across pictures of the video.

### Claim 3 (depends on 1)

The method of claim 1, wherein the reshaper model is a piecewise linear model,
wherein scale coefficients of the piecewise linear model are determined based on first variables which are determined based on the parameter information included in the adaptation parameter set and a second variable which is determined based on bit depth, and
wherein scaling parameters of the scaling process are derived based on reconstructed luma samples in the reshaped domain of a picture which the current video block belongs to and the chroma residual scaling factors.

### Claim 4 (depends on 1)

The method of claim 1, wherein the adaptation parameter set for the coding tool and another adaptation parameter set for adaptive loop filter information share a same network abstraction layer unit type.

### Claim 5 (depends on 1)

The method of claim 1, wherein the conversion comprises:
determining, for the luma component of the current video block, a first prediction block in the original domain from at least one reference picture using an unrefined motion information;
applying, at least based on the first prediction block, a motion information refinement process on the unrefined motion information to obtain a motion offset information;
generating a second prediction block from samples in the at least one reference picture using the motion offset information;
converting, during the forward mapping process, the second prediction block from the original domain to the reshaped domain in which samples of the second prediction block are mapped into the particular values;
generating reconstructed samples of the luma component of the current video block based on the second prediction block in the reshaped domain;
converting, during the inverse mapping process, the reconstructed samples of the luma component of the current video block from the reshaped domain into the original domain; and
applying a filtering process on the converted reconstructed samples in the original domain.

### Claim 6 (depends on 5)

The method of claim 5, wherein by applying the scaling process, the residual samples of the chroma component of the current video block are derived based on reconstructed samples in the reshaped domain of a luma component of a picture which the current video block belongs to.

### Claim 7 (depends on 1)

The method of claim 1, wherein whether to apply the scaling process on the residual samples of the chroma component of the current video block is based on a block dimension of the current video block.

### Claim 8 (depends on 7)

The method of claim 7, wherein in response to the chroma component of the current video block being coded with all zero coefficients, the scaling process is skipped.

### Claim 9 (depends on 1)

The method of claim 1, wherein the conversion comprises:
making a determination that a certain non-inter coding mode is used for the luma component of the current video block; and
performing the conversion by using the certain non-inter coding mode in the reshaped domain for the luma component to generate intermediate luma reconstruction samples of the luma component,
wherein the inverse mapping process is applied on the intermediate luma reconstruction samples to generate final luma reconstruction samples in the original domain, and
wherein whether to apply the scaling process on the residual samples of the chroma component of the current video block is determined.

### Claim 10 (depends on 9)

The method of claim 9, wherein the certain non-inter coding mode comprises a palette mode, an intra block copy mode, or a block-based delta pulse code modulation mode.

### Claim 11 (depends on 1)

The method of claim 1, wherein the conversion includes encoding the current video block into the bitstream.

### Claim 12 (depends on 1)

The method of claim 1, wherein the conversion includes decoding the current video block from the bitstream.

### Claim 13

An apparatus for processing video data comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to:
perform a conversion between a current video block of a video region of a video and a bitstream of the video,
wherein the conversion uses a coding tool in which the current video block is constructed based on at least one of:
1) A forward mapping process for a luma component of the current video block, in which samples of the luma component are converted from an original domain to a reshaped domain,
2) An inverse mapping process, which is an inverse operation of the forward mapping process to convert the samples of the luma component in the reshaped domain to the original domain, or
3) A scaling process, in which residual samples of a chroma component of the current video block are scaled before being used to reconstruct the chroma component of the current video block,
wherein an adaptation parameter set is included in the bitstream which comprises parameter information for the coding tool and a first identifier of the adaptation parameter set,
wherein the first identifier of the adaptation parameter set is fixed-length coded and has a range between 0 to M, wherein M is 2K−1, and
wherein the parameter information includes at least one of an indication of information of a reshaper model which is used in the forward mapping process to map the samples of the luma component into particular values or chroma residual scaling factors which are used in the scaling process to derive scaled residual samples of the chroma component.

### Claim 14 (depends on 13)

The apparatus of claim 13, wherein the adaptation parameter set is allowed to be shared across pictures of the video.

### Claim 15 (depends on 13)

The apparatus of claim 13, wherein the reshaper model is a piecewise linear model,
wherein scale coefficients of the piecewise linear model are determined based on first variables which are determined based on the parameter information included in the adaptation parameter set and a second variable which is determined based on bit depth, and
wherein scaling parameters of the scaling process are derived based on reconstructed luma samples in the reshaped domain of a picture which the current video block belongs to and the chroma residual scaling factors.

### Claim 16 (depends on 13)

The apparatus of claim 13, wherein the adaptation parameter set for the coding tool and another adaptation parameter set for adaptive loop filter information share a same network abstraction layer unit type.

### Claim 17 (depends on 13)

The apparatus of claim 13, wherein the conversion comprises:
determining, for the luma component of the current video block, a first prediction block in the original domain from at least one reference picture using an unrefined motion information;
applying, at least based on the first prediction block, a motion information refinement process on the unrefined motion information to obtain a motion offset information;
generating a second prediction block from samples in the at least one reference picture using the motion offset information;
converting, during the forward mapping process, the second prediction block from the original domain to the reshaped domain in which samples of the second prediction block are mapped into the particular values;
generating reconstructed samples of the luma component of the current video block based on the second prediction block in the reshaped domain;
converting, during the inverse mapping process, the reconstructed samples of the luma component of the current video block from the reshaped domain into the original domain; and
applying a filtering process on the converted reconstructed samples in the original domain.

### Claim 18 (depends on 17)

The apparatus of claim 17, wherein by applying the scaling process, the residual samples of the chroma component of the current video block are derived based on reconstructed samples in the reshaped domain of a luma component of a picture which the current video block belongs to.

### Claim 19 (depends on 13)

The apparatus of claim 13, wherein whether to apply the scaling process on the residual samples of the chroma component of the current video block is based on a block dimension of the current video block.

### Claim 20 (depends on 19)

The apparatus of claim 19, wherein in response to the chroma component of the current video block being coded with all zero coefficients, the scaling process is skipped.

### Claim 21 (depends on 13)

The apparatus of claim 13, wherein the conversion comprises:
making a determination that a certain non-inter coding mode is used for the luma component of the current video block; and
performing the conversion by using the certain non-inter coding mode in the reshaped domain for the luma component to generate intermediate luma reconstruction samples of the luma component, and
wherein the inverse mapping process is applied on the intermediate luma reconstruction samples to generate final luma reconstruction samples in the original domain, and
wherein whether to apply the scaling process on the residual samples of the chroma component of the current video block is determined.

### Claim 22 (depends on 21)

The apparatus of claim 21, wherein the certain non-inter coding mode comprises a palette mode, an intra block copy mode, or a block-based delta pulse code modulation mode.

### Claim 23

A non-transitory computer-readable storage medium storing instructions that cause a processor to:
perform a conversion between a current video block of a video region of a video and a bitstream of the video,
wherein the conversion uses a coding tool in which the current video block is constructed based on at least one of:
1) A forward mapping process for a luma component of the current video block, in which samples of the luma component are converted from an original domain to a reshaped domain,
2) An inverse mapping process, which is an inverse operation of the forward mapping process to convert the samples of the luma component in the reshaped domain to the original domain, or
3) A scaling process, in which residual samples of a chroma component of the current video block are scaled before being used to reconstruct the chroma component of the current video block,
wherein an adaptation parameter set is included in the bitstream which comprises parameter information for the coding tool and a first identifier of the adaptation parameter set,
wherein the first identifier of the adaptation parameter set is fixed-length coded and has a range between 0 to M, wherein M is 2K−1, and
wherein the parameter information includes at least one of an indication of information of a reshaper model which is used in the forward mapping process to map the samples of the luma component into particular values or chroma residual scaling factors which are used in the scaling process to derive scaled residual samples of the chroma component.

### Claim 24 (depends on 23)

The non-transitory computer-readable storage medium of claim 23, wherein the adaptation parameter set is allowed to be shared across pictures of the video,
wherein the reshaper model is a piecewise linear model,
wherein scale coefficients of the piecewise linear model are determined based on first variables which are determined based on the parameter information included in the adaptation parameter set and a second variable which is determined based on bit depth,
wherein scaling parameters of the scaling process are derived based on reconstructed luma samples in the reshaped domain of a picture which the current video block belongs to and the chroma residual scaling factors, and
wherein the adaptation parameter set for the coding tool and another adaptation parameter set for adaptive loop filter information share a same network abstraction layer unit type.

### Claim 25 (depends on 23)

The non-transitory computer-readable storage medium of claim 23, wherein the conversion comprises:
determining, for the luma component of the current video block, a first prediction block in the original domain from at least one reference picture using an unrefined motion information;
applying, at least based on the first prediction block, a motion information refinement process on the unrefined motion information to obtain a motion offset information;
generating a second prediction block from samples in the at least one reference picture using the motion offset information;
converting, during the forward mapping process, the second prediction block from the original domain to the reshaped domain in which samples of the second prediction block are mapped into the particular values;
generating reconstructed samples of the luma component of the current video block based on the second prediction block in the reshaped domain;
converting, during the inverse mapping process, the reconstructed samples of the luma component of the current video block from the reshaped domain into the original domain; and
applying a filtering process on the converted reconstructed samples in the original domain,
wherein by applying the scaling process, the residual samples of the chroma component of the current video block are derived based on reconstructed samples in the reshaped domain of a luma component of a picture which the current video block belongs to.

### Claim 26 (depends on 23)

The non-transitory computer-readable storage medium of claim 23, wherein whether to apply the scaling process on the residual samples of the chroma component of the current video block is based on a block dimension of the current video block, and
wherein in response to the chroma component of the current video block being coded with all zero coefficients, the scaling process is skipped.

### Claim 27 (depends on 23)

The non-transitory computer-readable storage medium of claim 23, wherein the conversion comprises:
making a determination that a certain non-inter coding mode is used for the luma component of the current video block; and
performing the conversion by using the certain non-inter coding mode in the reshaped domain for the luma component to generate intermediate luma reconstruction samples of the luma component,
wherein the inverse mapping process is applied on the intermediate luma reconstruction samples to generate final luma reconstruction samples in the original domain,
wherein whether to apply the scaling process on the residual samples of the chroma component of the current video block is determined, and
wherein the certain non-inter coding mode comprises a palette mode, an intra block copy mode, or a block-based delta pulse code modulation mode.

### Claim 28

A non-transitory computer-readable recording medium storing a bitstream of a video which is generated by a method performed by a video processing apparatus, wherein the method comprises:
generating the bitstream based on a current video block of a video region of the video,
wherein during the generating, a coding tool is used, in the coding tool, the current video block is constructed based on at least one of:
1) A forward mapping process for a luma component of the current video block, in which samples of the luma component are converted from an original domain to a reshaped domain,
2) An inverse mapping process, which is an inverse operation of the forward mapping process to convert the samples of the luma component in the reshaped domain to the original domain, or
3) A scaling process, in which residual samples of a chroma component of the current video block are scaled before being used to reconstruct the chroma component of the current video block,
wherein an adaptation parameter set is included in the bitstream which comprises parameter information for the coding tool and a first identifier of the adaptation parameter set,
wherein the first identifier of the adaptation parameter set is fixed-length coded and has a range between 0 to M, wherein M is 2K−1, and
wherein the parameter information includes at least one of an indication of information of a reshaper model which is used in the forward mapping process to map the samples of the luma component into particular values or chroma residual scaling factors which are used in the scaling process to derive scaled residual samples of the chroma component.

### Claim 29 (depends on 28)

The non-transitory computer-readable recording medium of claim 28, wherein the adaptation parameter set is allowed to be shared across pictures of the video,
wherein the reshaper model is a piecewise linear model,
wherein scale coefficients of the piecewise linear model are determined based on first variables which are determined based on the parameter information included in the adaptation parameter set and a second variable which is determined based on bit depth,
wherein scaling parameters of the scaling process are derived based on reconstructed luma samples in the reshaped domain of a picture which the current video block belongs to and the chroma residual scaling factors, and
wherein the adaptation parameter set for the coding tool and another adaptation parameter set for adaptive loop filter information share a same network abstraction layer unit type.

### Claim 30 (depends on 28)

The non-transitory computer-readable recording medium of claim 28, wherein during the generating, the method comprises:
determining, for the luma component of the current video block, a first prediction block in the original domain from at least one reference picture using an unrefined motion information;
applying, at least based on the first prediction block, a motion information refinement process on the unrefined motion information to obtain a motion offset information;
generating a second prediction block from samples in the at least one reference picture using the motion offset information;
converting, during the forward mapping process, the second prediction block from the original domain to the reshaped domain in which samples of the second prediction block are mapped into the particular values;
generating reconstructed samples of the luma component of the current video block based on the second prediction block in the reshaped domain;
converting, during the inverse mapping process, the reconstructed samples of the luma component of the current video block from the reshaped domain into the original domain; and
applying a filtering process on the converted reconstructed samples in the original domain,
wherein by applying the scaling process, the residual samples of the chroma component of the current video block are derived based on reconstructed samples in the reshaped domain of a luma component of a picture which the current video block belongs to.

### Claim 31 (depends on 28)

The non-transitory computer-readable recording medium of claim 28, wherein whether to apply the scaling process on the residual samples of the chroma component of the current video block is based on a block dimension of the current video block, and
wherein in response to the chroma component of the current video block being coded with all zero coefficients, the scaling process is skipped.

### Claim 32 (depends on 28)

The non-transitory computer-readable recording medium of claim 28, wherein during the generating, the method comprises:
making a determination that a certain non-inter coding mode is used for the luma component of the current video block; and
using the certain non-inter coding mode in the reshaped domain for the luma component to generate intermediate luma reconstruction samples of the luma component,
wherein the inverse mapping process is applied on the intermediate luma reconstruction samples to generate final luma reconstruction samples in the original domain,
wherein whether to apply the scaling process on the residual samples of the chroma component of the current video block is determined, and
wherein the certain non-inter coding mode comprises a palette mode, an intra block copy mode, or a block-based delta pulse code modulation mode.