# US 12,015,791 B2 · Video coding using multi-model linear model > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,015,791 B2 - Document ID: US12015791B2 - Status: Granted - Kind: B2 - Application number: 17/438,334 - Filed: 2020-03-11 - Published: 2022-11-10 - Granted: 2024-06-18 - Earliest claimed: 2019-03-12 - Assignees: Beijing Dajia Internet Information Technology Co., Ltd. - Applicants: Beijing Dajia Internet Information Technology Co., Ltd. - Inventors: Xianglin Wang; Hong-Jheng Jhu; Yi-Wen Chen - Web page: https://patentgrep.com/patent/US12015791B2 ## CPC - H04N 19/186: the unit being a colour or a chrominance component - H04N 19/593: involving spatial prediction techniques - H04N 19/105: Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction - H04N 19/132: Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking - H04N 19/176: the region being a block, e.g. a macroblock - H04N 19/182: the unit being a pixel - H04N 19/42: characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation (H04N19/635 takes precedence) - H04N 19/82: involving filtering within a prediction loop - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A computing device performs a method of decoding video data by receiving bitstream encoding a chroma block, a corresponding luma block, neighboring luma samples, and neighboring chroma samples; decoding the luma block, the plurality of neighboring luma samples, and the plurality of neighboring chroma samples; selecting a group of reference luma samples and a group of reference chroma samples; computing a threshold luma value from the plurality of reconstructed neighboring luma samples, and a threshold chroma value from the plurality of reconstructed neighboring chroma samples; determining a maximum luma value and a minimum luma value from the group of the reference luma samples; generating multi-model linear model (MMLM) including a first linear model between the minimum luma value and the threshold luma value, and a second linear model between the threshold luma value and the maximum luma value; and reconstructing the chroma block from the luma block using MMLM. ## Claims ### Claim 1 A method for reconstructing a chroma block of a video signal, comprising: receiving bitstream encoding the chroma block, a corresponding luma block, a plurality of neighboring luma samples surrounding the luma block, and a plurality of neighboring chroma samples surrounding the chroma block; decoding the luma block, the plurality of neighboring luma samples, and the plurality of neighboring chroma samples to obtain a plurality of reconstructed luma samples of the luma block, a plurality of reconstructed neighboring luma samples, and a plurality of reconstructed neighboring chroma samples, respectively; selecting, from the plurality of reconstructed neighboring luma samples and the plurality of reconstructed neighboring chroma samples, a group of reference luma samples and a group of reference chroma samples, wherein each reference luma sample corresponds to a respective reference chroma sample; where selecting the group of reference luma samples and the group of reference chroma samples includes selecting every other luma samples from the plurality of reconstructed neighboring luma samples and every other chroma samples from the plurality of reconstructed neighboring chroma samples; computing a threshold luma value from the plurality of reconstructed neighboring luma samples, and a threshold chroma value from the plurality of reconstructed neighboring chroma samples; determining a maximum luma value and a minimum luma value from the group of the reference luma samples, wherein the threshold luma value is between the minimum luma value and the maximum luma value; generating a multi-model linear model including a first linear model between the minimum luma value and the threshold luma value, and a second linear model between the threshold luma value and the maximum luma value; and reconstructing the chroma block from the luma block using the multi-model linear model; wherein generating the multi-model linear model includes: determining a first chroma value of a first reference chroma sample corresponding to a first reference luma sample having the maximum luma value, and a second chroma value of a second reference chroma sample corresponding to a second reference luma sample having the minimum luma value; and wherein the first linear model connects (minimum luma value, second chroma value) and (threshold luma value, threshold chroma value) and the second linear model connects (threshold luma value, threshold chroma value) and (maximum luma value, first chroma value). ### Claim 2 (depends on 1) The method of claim 1, wherein reconstructing the chroma block from the luma block using the multi-model linear model includes: for a respective chroma sample in the chroma block: determining a respective luma value of a respective luma sample in a decoded luma block that corresponds to a respective chroma sample; in accordance with a determination that the respective luma value is smaller than or equal to the threshold luma value: applying the first linear model to the respective luma value to obtain a respective chroma value; and in accordance with a determination that the respective luma value is greater than or equal to the threshold luma value: applying the second linear model to the respective luma value to obtain a respective chroma value. ### Claim 3 (depends on 1) The method of claim 1, wherein computing the threshold luma value includes finding an average luma value from the plurality of reconstructed neighboring luma samples, and computing the threshold chroma value includes finding an average chroma value from the plurality of reconstructed neighboring chroma samples. ### Claim 4 (depends on 1) The method of claim 1, wherein selecting the group of reference luma samples and the group of chroma reference samples includes determining an upper limit of a number of reference luma samples and reference chroma samples to be used. ### Claim 5 (depends on 1) The method of claim 1, further including: computing a second threshold luma value greater than the threshold luma value and a corresponding second threshold chroma value greater than the threshold chroma value, and wherein: the second linear model is applicable to luma values between the threshold luma value and the second threshold luma value, and a third linear model is applicable luma values between the second threshold luma value and the maximum luma value. ### Claim 6 (depends on 1) The method of claim 1, wherein computing the threshold luma value includes finding a weighted average luma value between the maximum luma value and the minimum luma value from the plurality of reconstructed neighboring luma samples, and computing the threshold chroma value includes finding a weighted average chroma value between the maximum chroma value and the minimum chroma value from the plurality of reconstructed neighboring chroma samples. ### Claim 7 (depends on 1) The method of claim 1, wherein computing the threshold luma value includes finding an average luma value from the plurality of reconstructed luma samples of the luma block. ### Claim 8 (depends on 1) The method of claim 1, wherein reconstructing the chroma block from the luma block using the multi-model linear model includes: for a respective block of chroma samples in the chroma block: determining a respective average luma value of a respective block of luma samples in a decoded luma block that corresponds to a respective block of chroma samples; in accordance with a determination that the respective average luma value is smaller than or equal to the threshold luma value: applying the first linear model to each luma value in the respective block of luma samples to obtain a respective chroma value in the respective block of chroma samples; and in accordance with a determination that the respective average luma value is greater than the threshold luma value: applying the second linear model to each luma value in the respective block of luma samples to obtain a respective chroma value in the respective block of chroma samples. ### Claim 9 A computing device including one or more processors, memory and a plurality of programs stored in the memory, wherein the programs, when executed by the one or more processors, cause the computing device to perform operations including: receiving bitstream encoding the chroma block, a corresponding luma block, a plurality of neighboring luma samples surrounding the luma block, and a plurality of neighboring chroma samples surrounding the chroma block; decoding the luma block, the plurality of neighboring luma samples, and the plurality of neighboring chroma samples to obtain a plurality of reconstructed luma samples of the luma block, a plurality of reconstructed neighboring luma samples, and a plurality of reconstructed neighboring chroma samples, respectively; selecting, from the plurality of reconstructed neighboring luma samples and the plurality of reconstructed neighboring chroma samples, a group of reference luma samples and a group of reference chroma samples, wherein each reference luma sample corresponds to a respective reference chroma sample; where selecting the group of reference luma samples and the group of reference chroma samples includes selecting every other luma samples from the plurality of reconstructed neighboring luma samples and every other chroma samples from the plurality of reconstructed neighboring chroma samples; computing a threshold luma value from the plurality of reconstructed neighboring luma samples, and a threshold chroma value from the plurality of reconstructed neighboring chroma samples; determining a maximum luma value and a minimum luma value from the group of the reference luma samples, wherein the threshold luma value is between the minimum luma value and the maximum luma value; generating a multi-model linear model including a first linear model between the minimum luma value and the threshold luma value, and a second linear model between the threshold luma value and the maximum luma value; and reconstructing the chroma block from the luma block using the multi-model linear model; wherein the computing device generates the multi-model linear model by perform operations including: determining a first chroma value of a first reference chroma sample corresponding to a first reference luma sample having the maximum luma value, and a second chroma value of a second reference chroma sample corresponding to a second reference luma sample having the minimum luma value; and wherein the first linear model connects (minimum luma value, second chroma value) and (threshold luma value, threshold chroma value) and the second linear model connects (threshold luma value, threshold chroma value) and (maximum luma value, first chroma value). ### Claim 10 (depends on 9) The computing device of claim 9, wherein the computing device reconstructs the chroma block from the luma block using the multi-model linear model by perform operations including: for a respective chroma sample in the chroma block: determining a respective luma value of a respective luma sample in a decoded luma block that corresponds to a respective chroma sample; in accordance with a determination that the respective luma value is smaller than or equal to the threshold luma value: applying the first linear model to the respective luma value to obtain a respective chroma value; and in accordance with a determination that the respective luma value is greater than or equal to the threshold luma value: applying the second linear model to the respective luma value to obtain a respective chroma value. ### Claim 11 (depends on 9) The computing device of claim 9, wherein the computing device computes the threshold luma value includes finding an average luma value from the plurality of reconstructed neighboring luma samples, and computes the threshold chroma value includes finding an average chroma value from the plurality of reconstructed neighboring chroma samples. ### Claim 12 (depends on 9) The computing device of claim 9, wherein the computing device selects the group of reference luma samples and the group of chroma reference samples by perform operations including: determining an upper limit of a number of reference luma samples and reference chroma samples to be used. ### Claim 13 (depends on 9) The computing device of claim 9, the computing device is configured to perform further operations including: computing a second threshold luma value greater than the threshold luma value and a corresponding second threshold chroma value greater than the threshold chroma value, and wherein: the second linear model is applicable to luma values between the threshold luma value and the second threshold luma value, and a third linear model is applicable luma values between the second threshold luma value and the maximum luma value. ### Claim 14 (depends on 9) The computing device of claim 9, wherein the computing device computes the threshold luma value includes finding a weighted average luma value between the maximum luma value and the minimum luma value from the plurality of reconstructed neighboring luma samples, and computes the threshold chroma value includes finding a weighted average chroma value between the maximum chroma value and the minimum chroma value from the plurality of reconstructed neighboring chroma samples. ### Claim 15 (depends on 9) The computing device of claim 9, wherein the computing device computes the threshold luma value by perform operations including finding an average luma value from the plurality of reconstructed luma samples of the luma block. ### Claim 16 A non-transitory computer readable storage medium storing a plurality of programs for execution by a computing device having one or more processors, wherein the programs, when executed by the one or more processors, cause the computing device to perform operations including: receiving bitstream encoding the chroma block, a corresponding luma block, a plurality of neighboring luma samples surrounding the luma block, and a plurality of neighboring chroma samples surrounding the chroma block; decoding the luma block, the plurality of neighboring luma samples, and the plurality of neighboring chroma samples to obtain a plurality of reconstructed luma samples of the luma block, a plurality of reconstructed neighboring luma samples, and a plurality of reconstructed neighboring chroma samples, respectively; selecting, from the plurality of reconstructed neighboring luma samples and the plurality of reconstructed neighboring chroma samples, a group of reference luma samples and a group of reference chroma samples, wherein each reference luma sample corresponds to a respective reference chroma sample; where selecting the group of reference luma samples and the group of reference chroma samples includes selecting every other luma samples from the plurality of reconstructed neighboring luma samples and every other chroma samples from the plurality of reconstructed neighboring chroma samples; computing a threshold luma value from the plurality of reconstructed neighboring luma samples, and a threshold chroma value from the plurality of reconstructed neighboring chroma samples; determining a maximum luma value and a minimum luma value from the group of the reference luma samples, wherein the threshold luma value is between the minimum luma value and the maximum luma value; generating a multi-model linear model including a first linear model between the minimum luma value and the threshold luma value, and a second linear model between the threshold luma value and the maximum luma value; and reconstructing the chroma block from the luma block using the multi-model linear model; wherein generating the multi-model linear model includes: determining a first chroma value of a first reference chroma sample corresponding to a first reference luma sample having the maximum luma value, and a second chroma value of a second reference chroma sample corresponding to a second reference luma sample having the minimum luma value; and wherein the first linear model connects (minimum luma value, second chroma value) and (threshold luma value, threshold chroma value) and the second linear model connects (threshold luma value, threshold chroma value) and (maximum luma value, first chroma value).