# US 12,015,784 B2 · Multi-models for intra prediction

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,015,784 B2
- Document ID: US12015784B2
- Status: Granted
- Kind: B2
- Application number: 18/345,608
- Filed: 2023-06-30
- Published: 2023-10-26
- Granted: 2024-06-18
- Earliest claimed: 2018-11-06
- Assignees: Beijing Bytedance Network Technology Co., Ltd.; Bytedance Inc.
- Applicants: Beijing Bytedance Network Technology Co., Ltd.; Bytedance Inc.
- Inventors: Kai Zhang; Li Zhang; Hongbin Liu; Jizheng Xu; Yue Wang
- Web page: https://patentgrep.com/patent/US12015784B2

## CPC

- H04N 19/149: by estimating the code amount by means of a model, e.g. mathematical model or statistical model
- H04N 19/105: Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
- H04N 19/11: among a plurality of spatial predictive coding modes
- H04N 19/132: Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
- H04N 19/159: Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
- H04N 19/176: the region being a block, e.g. a macroblock
- H04N 19/184: the unit being bits, e.g. of the compressed video stream
- H04N 19/186: the unit being a colour or a chrominance component
- H04N 19/189: characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding
- H04N 19/30: using hierarchical techniques, e.g. scalability (H04N19/63 takes precedence)
- H04N 19/42: characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation (H04N19/635 takes precedence)
- H04N 19/50: using predictive coding (H04N19/61 takes precedence)
- H04N 19/593: involving spatial prediction techniques
- H04N 19/70: characterised by syntax aspects related to video coding, e.g. related to compression standards
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A method for video processing is provided. The method includes determining, for a conversion between a current video block of a video that is a chroma block and a coded representation of the video, a final prediction P(x, y) of a chroma sample at a position (x, y) in the current video block as a combination of prediction results of multiple cross-component linear models (MCCLMs), wherein the MCCLMs are selected based on the position (x, y) of the chroma sample; and performing the conversion based on the final prediction.

## Claims

### Claim 1

A method of processing video data, comprising:
determining, for a conversion between a current video block of a video that is a chroma block and a bitstream of the video, values of parameters of a cross-component linear model (CCLM) in a CCLM mode based on two chroma values denoted as minC and maxC and two luma values denoted as minY and maxY;
deriving, using the CCLM, predicted samples of the current video block based on reconstructed samples of a luma block corresponding to the current video block and the parameters of the CCLM; and
performing the conversion based on the predicted samples of the current video block,
wherein when maxY is not equal to minY, then one of the parameters is further determined based on a look-up table using an index derived based on a difference between maxY and minY,
wherein when maxY is equal to minY, then the predicted samples of the current video block are based on minC and not based on minY, maxY and maxC,
wherein maxY and minY are derived based on corresponding luma samples of selected chroma samples, wherein the selected chroma samples are selected from a group of neighboring chroma samples based on positions of the neighboring chroma samples and the CCLM mode of the current video block, wherein the positions of the neighboring chroma samples are derived based on distances from a top-left sample of the current video block to the neighboring chroma samples,
wherein minC and maxC are derived based on two or more chroma selected samples from the neighboring chroma samples of the current video block, wherein the two or more chroma selected samples are selected based on the CCLM mode of the current video block and availabilities of the neighboring chroma samples,
wherein the CCLM mode of the current video block is one of a first CCLM mode that derives the parameters of CCLM based on left neighboring chroma samples and above neighboring chroma samples, a second CCLM mode that derives the parameters of CCLM based on the left neighboring chroma samples and below-left neighboring samples, and a third CCLM mode that derives the parameters of CCLM based on the above neighboring chroma samples and above-right neighboring chroma sample, wherein a width and a height of the current video block is W and H, respectively,
wherein the two or more chroma samples are selected further based on the W and/or the H, and wherein exactly two chroma samples are selected from the above neighboring chroma samples in response to only the above neighboring chroma samples being available, the CCLM mode of the current video block being the first CCLM mode, and the W being equal to 2,
wherein positions of the two or more selected chroma selected samples derived based on a first position offset value (F) and a step value (S), and wherein F and S are derived at least based on availabilities of the neighboring chroma samples of the current video block and a dimension of the current video block,
wherein a Floor operation comprises F=Floor (M/2i), wherein M is a number of the neighboring chroma samples used to derive the selected chroma samples in horizontal direction, or F=Floor (N/2i), wherein N is a number of the neighboring chroma samples used to derive the selected chroma samples in vertical direction, i is equal to 2 or 3, and the Floor operation is used to obtain an integer part of a number, and
wherein a Max operation comprises S=Max(1, Floor (M/2j)), or S=Max(1, Floor (N/2j)), j is equal to 1 or 2, and the Max operation is used to obtain a maximum of multiple numbers.

### Claim 2 (depends on 1)

The method of claim 1, wherein M and N are both less than or equal to W+H, and determined based on the CCLM mode of the current video block, and
wherein in response to the CCLM mode being the first CCLM mode and the above neighboring chroma samples being available, M is equal to W.

### Claim 3 (depends on 1)

The method of claim 1, wherein in response to selecting two chroma samples in the horizontal direction, positions of the two selected chroma samples in the horizontal direction are Floor(M/4) and Floor (M/4)+Floor (M/2),
wherein in response to selecting two chroma samples in the vertical direction, positions of the two selected chroma samples in the vertical direction are Floor(N/4) and Floor (N/4)+Floor(N/2).

### Claim 4 (depends on 1)

The method of claim 1, wherein in response to selecting four chroma samples in the horizontal direction, positions of the four selected chroma samples in the horizontal direction are Floor(M/8), Floor (M/8)+Floor (M/4), Floor (M/8)+2*Floor (M/4), Floor (M/8)+3*Floor (M/4), and
wherein in response to selecting four chroma samples in the vertical direction, positions of the four selected chroma samples in the vertical direction are Floor(N/8), Floor(N/8)+Floor (N/4), Floor(N/8)+2*Floor (N/4), Floor(N/8)+3*Floor (N/4).

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
determining, based on a color format of the current video block, whether the luma samples are downsampled to derive the values of parameters of the cross-component linear model,
wherein the luma samples are downsampled in response to the color format being 4:2:0 or 4:2:2.

### Claim 6 (depends on 1)

The method of claim 1, wherein when the CCLM mode is the second CCLM mode, exactly two chroma samples are selected from the left neighboring chroma samples in response to only the left neighboring chroma samples being available and the height of the current video block being equal to 2, and
wherein when the CCLM mode is the second CCLM mode, exactly four chroma samples are selected from the left neighboring chroma samples in response to only the left neighboring chroma samples being available and the height of the current video block being larger than 2.

### Claim 7 (depends on 1)

The method of claim 1, wherein when the CCLM mode is the third CCLM mode, exactly two chroma samples are selected from the above neighboring chroma samples in response to only the above neighboring chroma samples being available and the width of the current video block being equal to 2, and
wherein when the CCLM mode is the third CCLM mode, exactly four chroma samples are selected from the above neighboring chroma samples in response to only the above neighboring chroma samples being available and the width of the current video block being larger than 2.

### Claim 8 (depends on 1)

The method of claim 1, wherein the parameters of the CCLM are derived based on a value of Floor (Log2(MAXY-MINY)), and wherein Floor(x) is a floor function that outputs an integer portion of x.

### Claim 9 (depends on 1)

The method of claim 1, wherein the determining of the parameters is further based on a two-point method, and
wherein at least one sample of the group of neighboring chroma samples that is located beyond 2×W above neighboring chroma samples or 2×H left neighboring chroma samples.

### Claim 10 (depends on 1)

The method of claim 1, wherein in a case that maxY is equal to minY, the predicted samples of the current video block are based on a specific chroma value minC, and wherein minC is derived from the neighboring chroma samples of the current video block, wherein the predicted samples of the current video block are set to minC.

### Claim 11 (depends on 1)

The method of claim 1, wherein in response to chroma samples and two corresponding luma samples mapped with two original indices being used to derive maxY, maxC, minY, and minC, applying a padding operation to generate two padded chroma samples and two padded luma samples, and
wherein the two original indices are denoted as S0 and S1, and two mapped indices with which the padded chroma samples and the two padded luma samples are mapped are denoted as S2 and S3, and wherein the padding operation is done as follows orderly:
1) The chroma sample and the corresponding luma sample mapped with S0 is copied to the chroma sample and the corresponding luma sample mapped with S3, respectively;
2) The chroma sample and the corresponding luma sample mapped with S1 is copied to the chroma sample and the corresponding luma sample mapped with S2, respectively;
3) The chroma sample and the corresponding luma sample mapped with S1 is copied to the chroma sample and the corresponding luma sample mapped with S0, respectively;
4) The chroma sample and the corresponding luma sample mapped with S3 is copied to the chroma sample and the corresponding luma sample mapped with S1, respectively.

### Claim 12 (depends on 1)

The method of claim 1, wherein in response to four chroma samples and four corresponding luma samples mapped with four indices being used to derive maxY, maxC, minY, and minC, the four indices are divided into two arrays G0 and G1, each array includes two indices,
wherein G0[0]=S0, G0[1]=S2, G1[0]=S1, and G1[1]=S3, and wherein each of S0, S1, S2 and S3 represent one index, which is mapped with a chroma sample and a corresponding luma sample,
wherein the method further comprises applying a first comparison between the luma samples of G0[0] and the luma samples of G0[1], and selectively swapping G0[0] and G0[1] based on the first comparison, and
wherein the method further comprises swapping G0[0] and G0[1] in response to the luma samples of G0[0] is larger than the luma samples of G0[1].

### Claim 13 (depends on 1)

The method of claim 1, wherein when maxY is equal to minY, other intra prediction mode than the cross-component linear model in the CCLM mode is used, and
wherein the parameters of the cross-component linear model are derived by excluding a division operation; or, wherein the parameters of the cross-component linear model are derived by using an operation without the look up table, the operation excluding the division operation.

### Claim 14 (depends on 1)

The method of claim 1, further comprising determining two or more luma samples corresponding the two or more chroma samples,
wherein in response to four luma samples being determined, two maximum values of the four luma samples are averaged to derive a first averaging luma value and two minimum values of the four luma samples are averaged to derive a second averaging luma value, respectively, and
wherein the first averaging luma value and the second averaging luma value are used to derive the parameters of CCLM.

### Claim 15 (depends on 1)

The method of claim 1, in response to the neighboring chroma samples of the current video block being unavailable, the prediction values of the current video block are set to a default value, and
wherein the default value is equal to 1<<(BitDepth-1), wherein BitDepth represents a bit-depth of the chroma samples.

### Claim 16 (depends on 1)

The method of claim 1, wherein the conversion includes encoding the current video block into the bitstream.

### Claim 17 (depends on 1)

The method of claim 1, wherein the conversion includes decoding the current video block from the bitstream.

### Claim 18

An apparatus for processing video data comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to:
determine, for a conversion between a current video block of a video that is a chroma block and a bitstream of the video, values of parameters of a cross-component linear model (CCLM) in a CCLM mode based on two chroma values denoted as minC and maxC and two luma values denoted as minY and maxY;
derive, using the CCLM, predicted samples of the current video block based on reconstructed samples of a luma block corresponding to the current video block and the parameters of the CCLM; and
perform the conversion based on the predicted samples of the current video block,
wherein when maxY is not equal to minY, then one of the parameters is further determined based on a look-up table using an index derived based on a difference between maxY and minY,
wherein when maxY is equal to minY, then the predicted samples of the current video block are based on minC and not based on minY, maxY and maxC,
wherein maxY and minY are derived based on corresponding luma samples of selected chroma samples, wherein the selected chroma samples are selected from a group of neighboring chroma samples based on positions of the neighboring chroma samples and the CCLM mode of the current video block, wherein the positions of the neighboring chroma samples are derived based on distances from a top-left sample of the current video block to the neighboring chroma samples,
wherein minC and maxC are derived based on two or more chroma selected samples from the neighboring chroma samples of the current video block, wherein the two or more chroma selected samples are selected based on the CCLM mode of the current video block and availabilities of the neighboring chroma samples,
wherein the CCLM mode of the current video block is one of a first CCLM mode that derives the parameters of CCLM based on left neighboring chroma samples and above neighboring chroma samples, a second CCLM mode that derives the parameters of CCLM based on the left neighboring chroma samples and below-left neighboring samples, and a third CCLM mode that derives the parameters of CCLM based on the above neighboring chroma samples and above-right neighboring chroma sample, wherein a width and a height of the current video block is W and H, respectively,
wherein the two or more chroma samples are selected further based on the W and/or the H, and wherein exactly two chroma samples are selected from the above neighboring chroma samples in response to only the above neighboring chroma samples being available, the CCLM mode of the current video block being the first CCLM mode, and the W being equal to 2,
wherein positions of the two or more selected chroma selected samples derived based on a first position offset value (F) and a step value (S), and wherein F and S are derived at least based on availabilities of the neighboring chroma samples of the current video block and a dimension of the current video block,
wherein a Floor operation comprises F=Floor (M/2i), wherein M is a number of the neighboring chroma samples used to derive the selected chroma samples in horizontal direction, or F=Floor (N/2i), wherein N is a number of the neighboring chroma samples used to derive the selected chroma samples in vertical direction, i is equal to 2 or 3, and the Floor operation is used to obtain an integer part of a number, and
wherein a Max operation comprises S=Max(1, Floor (M/2j)), or S=Max(1, Floor (N/2j)), j is equal to 1 or 2, and the Max operation is used to obtain a maximum of multiple numbers.

### Claim 19

A non-transitory computer-readable storage medium storing instructions that cause a processor to:
determine, for a conversion between a current video block of a video that is a chroma block and a bitstream of the video, values of parameters of a cross-component linear model (CCLM) in a CCLM mode based on two chroma values denoted as minC and maxC and two luma values denoted as minY and maxY;
derive, using the CCLM, predicted samples of the current video block based on reconstructed samples of a luma block corresponding to the current video block and the parameters of the CCLM; and
perform the conversion based on the predicted samples of the current video block,
wherein when maxY is not equal to minY, then one of the parameters is further determined based on a look-up table using an index derived based on a difference between maxY and minY,
wherein when maxY is equal to minY, then the predicted samples of the current video block are based on minC and not based on minY, maxY and maxC,
wherein maxY and minY are derived based on corresponding luma samples of selected chroma samples, wherein the selected chroma samples are selected from a group of neighboring chroma samples based on positions of the neighboring chroma samples and the CCLM mode of the current video block, wherein the positions of the neighboring chroma samples are derived based on distances from a top-left sample of the current video block to the neighboring chroma samples,
wherein minC and maxC are derived based on two or more chroma selected samples from the neighboring chroma samples of the current video block, wherein the two or more chroma selected samples are selected based on the CCLM mode of the current video block and availabilities of the neighboring chroma samples,
wherein the CCLM mode of the current video block is one of a first CCLM mode that derives the parameters of CCLM based on left neighboring chroma samples and above neighboring chroma samples, a second CCLM mode that derives the parameters of CCLM based on the left neighboring chroma samples and below-left neighboring samples, and a third CCLM mode that derives the parameters of CCLM based on the above neighboring chroma samples and above-right neighboring chroma sample, wherein a width and a height of the current video block is W and H, respectively,
wherein the two or more chroma samples are selected further based on the W and/or the H, and wherein exactly two chroma samples are selected from the above neighboring chroma samples in response to only the above neighboring chroma samples being available, the CCLM mode of the current video block being the first CCLM mode, and the W being equal to 2,
wherein positions of the two or more selected chroma selected samples derived based on a first position offset value (F) and a step value (S), and wherein F and S are derived at least based on availabilities of the neighboring chroma samples of the current video block and a dimension of the current video block,
wherein a Floor operation comprises F=Floor (M/2i), wherein M is a number of the neighboring chroma samples used to derive the selected chroma samples in horizontal direction, or F=Floor (N/2i), wherein N is a number of the neighboring chroma samples used to derive the selected chroma samples in vertical direction, i is equal to 2 or 3, and the Floor operation is used to obtain an integer part of a number, and
wherein a Max operation comprises S=Max(1, Floor (M/2j)), or S=Max(1, Floor (N/2j)), j is equal to 1 or 2, and the Max operation is used to obtain a maximum of multiple numbers.

### Claim 20

A non-transitory computer-readable recording medium storing a bitstream of a video which is generated by a method performed by a video processing apparatus, wherein the method comprises:
determining, for a current video block of the video that is a chroma block, values of parameters of a cross-component linear model (CCLM) in a CCLM mode based on two chroma values denoted as minC and maxC and two luma values denoted as minY and maxY;
deriving, using the CCLM, predicted samples of the current video block based on reconstructed samples of a luma block corresponding to the current video block and the parameters of the CCLM; and
generating the bitstream based on the predicted samples of the current video block,
wherein when maxY is not equal to minY, then one of the parameters is further determined based on a look-up table using an index derived based on a difference between maxY and minY,
wherein when maxY is equal to minY, then the predicted samples of the current video block are based on minC and not based on minY, maxY and maxC,
wherein maxY and minY are derived based on corresponding luma samples of selected chroma samples, wherein the selected chroma samples are selected from a group of neighboring chroma samples based on positions of the neighboring chroma samples and the CCLM mode of the current video block, wherein the positions of the neighboring chroma samples are derived based on distances from a top-left sample of the current video block to the neighboring chroma samples, wherein minC and maxC are derived based on two or more chroma selected samples from the neighboring chroma samples of the current video block, wherein the two or more chroma selected samples are selected based on the CCLM mode of the current video block and availabilities of the neighboring chroma samples;
wherein the CCLM mode of the current video block is one of a first CCLM mode that derives the parameters of CCLM based on left neighboring chroma samples and above neighboring chroma samples, a second CCLM mode that derives the parameters of CCLM based on the left neighboring chroma samples and below-left neighboring samples, and a third CCLM mode that derives the parameters of CCLM based on the above neighboring chroma samples and above-right neighboring chroma sample, wherein a width and a height of the current video block is W and H, respectively,
wherein the two or more chroma samples are selected further based on the W and/or the H, and wherein exactly two chroma samples are selected from the above neighboring chroma samples in response to only the above neighboring chroma samples being available, the CCLM mode of the current video block being the first CCLM mode, and the W being equal to 2,
wherein positions of the two or more selected chroma selected samples derived based on a first position offset value (F) and a step value (S), and wherein F and S are derived at least based on availabilities of the neighboring chroma samples of the current video block and a dimension of the current video block,
wherein a Floor operation comprises F=Floor (M/2i), wherein M is a number of the neighboring chroma samples used to derive the selected chroma samples in horizontal direction, or F=Floor (N/2i), wherein N is a number of the neighboring chroma samples used to derive the selected chroma samples in vertical direction, i is equal to 2 or 3, and the Floor operation is used to obtain an integer part of a number,
wherein a Max operation comprises S=Max(1, Floor (M/2j)), or S=Max(1, Floor (N/2j)), j is equal to 1 or 2, and the Max operation is used to obtain a maximum of multiple numbers.