# US 12,192,480 B2 · Weight derivation of multiple reference line for intra prediction fusion > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,192,480 B2 - Document ID: US12192480B2 - Status: Granted - Kind: B2 - Application number: 17/984,926 - Filed: 2022-11-10 - Published: 2024-01-18 - Granted: 2025-01-07 - Earliest claimed: 2022-07-13 - Assignees: Tencent America LLC - Applicants: Tencent America LLC - Inventors: Lien-Fei Chen; Xin Zhao; Guichun Li; Shan Liu - Web page: https://patentgrep.com/patent/US12192480B2 ## CPC - H04N 19/159: Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction - H04N 19/105: Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction - H04N 19/11: among a plurality of spatial predictive coding modes - H04N 19/132: Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking - H04N 19/176: the region being a block, e.g. a macroblock - H04N 19/593: involving spatial prediction techniques - H04N 19/70: characterised by syntax aspects related to video coding, e.g. related to compression standards - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract Aspects of the disclosure provide a method and an apparatus for video decoding. The apparatus includes processing circuitry. The processing circuitry applies an intra prediction mode with a multiple reference line (MRL) prediction to a current block. For each weight candidate combination including a respective first weight candidate and a respective second weight candidate in a plurality of weight candidate combinations, a subset of samples in the current block is predicted using intra prediction fusion that is based on a first reference line weighted by the respective first weight candidate and a second reference line weighted by the respective second weight candidate. The subset of samples includes top samples in a top-most row and left samples in a left-most column. A gradient cost is determined based on the predicted subset of samples and reconstructed samples outside the current block. A weight candidate combination is selected based on the gradient costs. ## Claims ### Claim 1 A method of video decoding in a decoder, comprising: receiving a coded bitstream comprising a current block in a current picture; obtaining, from the coded bitstream, prediction information indicating that an intra prediction mode with a multiple reference line (MRL) prediction is applied to the current block, the current block being predicted based on a first reference line and a second reference line, wherein a first weight candidate is applied to the first reference line and a second weight candidate is applied to the second reference line and the first reference line and the second reference line are different; obtaining from the coded bitstream a plurality of weight candidate combinations, each weight candidate combination including a respective first weight candidate and a respective second weight candidate; for each weight candidate combination, predicting a subset of samples in the current block using intra prediction fusion that is based on the first reference line weighted by the respective first weight candidate and the second reference line weighted by the respective second weight candidate, wherein the subset of samples includes (i) top samples in a top-most row in the current block and (ii) left samples in a left-most column in the current block; and determining a gradient cost based on the predicted subset of samples in the current block and reconstructed samples outside the current block, the reconstructed samples outside the current block including samples that neighbor the predicted subset of samples in the current block; and selecting a weight candidate combination based on the determined gradient cost corresponding to each weight candidate combination. ### Claim 2 (depends on 1) The method of claim 1, wherein: the first reference line includes first reference samples that are N1 rows or N1 columns away from the current block; the second reference line includes second reference samples that are N2 rows or N2 columns away from the current block; and N1 and N2 are different integers that are larger than or equal to 0. ### Claim 3 (depends on 1) The method of claim 1, wherein the predicting the subset of samples comprises: for each weight candidate combination in the plurality of weight candidate combinations and for one of the subset of samples, determining a first prediction value based on one or more first reference samples of the first reference line using the intra prediction mode; determining a second prediction value based on one or more second reference samples of the second reference line using the intra prediction mode; and predicting the one of the subset of samples based on the first prediction value modified by the first weight candidate, the second prediction value modified by the second weight candidate, and a residual of the one of the subset of samples. ### Claim 4 (depends on 3) The method of claim 3, wherein the reconstructed samples outside the current block include reconstructed samples that are not adjacent to the predicted subset of samples. ### Claim 5 (depends on 1) The method of claim 1, wherein the selecting the weight candidate combination comprises: selecting the weight candidate combination to be one of the plurality of weight candidate combinations that corresponds to the smallest gradient cost in the determined gradient costs. ### Claim 6 (depends on 1) The method of claim 1, wherein the selecting the weight candidate combination comprises: reordering the plurality of weight candidate combinations based on the determined gradient costs; and determining the weight candidate combination based on an index and the reordered plurality of weight candidate combinations. ### Claim 7 (depends on 6) The method of claim 6, wherein the index is signaled in a high-level syntax. ### Claim 8 A method of video decoding in a decoder, comprising: decoding prediction information of a current block in a current picture from a coded bitstream, the prediction information indicating an intra prediction mode being applied to the current block with a multiple reference line (MRL) prediction; receiving an MRL index i from the coded bitstream indicating that an ith entry in a MRL list corresponds to a first reference line; determining that an (i+1)th entry in the MRL list corresponds to a second reference line; and reconstructing a sample in the current block using intra prediction fusion that is based on multiple reference lines of the current block, the first reference line corresponding to the ith entry in the MRL list and the second reference line corresponding to the (i+1)th entry in the MRL list, the first reference line and the second reference line not being spatially adjacent. ### Claim 9 (depends on 8) The method of claim 8, wherein the reconstructing the sample comprises: determining a first prediction value based on one or more first reference samples in the first reference line using the intra prediction mode; determining a second prediction value based on one or more second reference samples in the second reference line using the intra prediction mode; and predicting the sample based on a weighted average of the first prediction value and the second prediction value. ### Claim 10 (depends on 8) The method of claim 8, wherein the reconstructing the sample comprises: reconstructing the sample based on the predicted sample and a residual of the sample. ### Claim 11 (depends on 9) The method of claim 9, wherein the MRL list is {1, 3, 5, 7, 12} corresponding to reference lines 1, 3, 5, 7, and 12, respectively, each of the reference lines 1, 3, 5, 7, and 12 being 1, 3, 5, 7, and 12 rows and/or columns away from the current block, respectively; the MRL index i being 0, 2, 3, or 4 corresponds to the reference line 1, 5, 7, or 12, respectively; in response to the MRL index i being 0, the first reference line and the second reference line are the reference lines 1 and 3; in response to the MRL index i being 2, the first reference line and the second reference line are the reference lines 5 and 7; in response to the MRL index i being 3, the first reference line and the second reference line are the reference lines 7 and 12; and in response to the MRL index i being 4, the first reference line and the second reference line are the reference lines 12 and 1. ### Claim 12 (depends on 8) The method of claim 8, wherein a first weight associated with the first reference line and a second weight associated with the second reference line depend on the first reference line and the second reference line; and the reconstructing the sample includes reconstructing the sample in the current block based on the first reference line, the second reference line, the first weight, and the second weight. ### Claim 13 (depends on 8) The method of claim 8, wherein a first weight associated with the first reference line and a second weight associated with the second reference line depend on a distance between the first reference line and the second reference line, the distance being a number of rows and/or columns between the first reference line and the second reference line. ### Claim 14 (depends on 8) The method of claim 8, wherein a first weight associated with the first reference line depends on a distance between the first reference line and a reference line 0 that is adjacent to the current block, the distance being proportional to the MRL index i. ### Claim 15 A method of video encoding, comprising: encoding prediction information indicating that an intra prediction mode with multiple reference line (MRL) prediction is applied to a current block in a current picture, the current block being predicted based on a first reference line and a second reference line that is different from the first reference line, a first weight candidate being applied to the first reference line, a second weight candidate being applied to the second reference line, each weight candidate combination in a plurality of weight candidate combinations including a respective first weight candidate and a respective second weight candidate, wherein for each weight candidate combination in the plurality of weight candidate combinations, a subset of samples in the current block is predicted using intra prediction fusion that is based on the first reference line weighted by the respective first weight candidate and the second reference line weighted by the respective second weight candidate, the subset of samples including (i) top samples in a top-most row in the current block and (ii) left samples in a left-most column in the current block; and a gradient cost is determined based on the predicted subset of samples in the current block and reconstructed samples outside the current block, the reconstructed samples outside the current block including samples that neighbor the predicted subset of samples in the current block; and a weight candidate combination is selected based on the determined gradient cost corresponding to each weight candidate combination. ### Claim 16 (depends on 15) The method of claim 15, wherein: the first reference line includes first reference samples that are N1 rows or N1 columns away from the current block; the second reference line includes second reference samples that are N2 rows or N2 columns away from the current block; and N1 and N2 are different integers that are larger than or equal to 0. ### Claim 17 (depends on 15) The method of claim 15, wherein the subset of samples is predicted by for each weight candidate combination in the plurality of weight candidate combinations and for one of the subset of samples, determining a first prediction value based on one or more first reference samples of the first reference line using the intra prediction mode; determining a second prediction value based on one or more second reference samples of the second reference line using the intra prediction mode; and predicting the one of the subset of samples based on the first prediction value modified by the first weight candidate, the second prediction value modified by the second weight candidate, and a residual of the one of the subset of samples. ### Claim 18 (depends on 17) The method of claim 17, wherein the reconstructed samples outside the current block include reconstructed samples that are not adjacent to the predicted subset of samples. ### Claim 19 (depends on 15) The method of claim 15, wherein the weight candidate combination is selected to be one of the plurality of weight candidate combinations that corresponds to the smallest gradient cost in the determined gradient costs. ### Claim 20 (depends on 15) The method of claim 15, wherein the weight candidate combination is selected by reordering the plurality of weight candidate combinations based on the determined gradient costs; and determining the weight candidate combination based on an index and the reordered plurality of weight candidate combinations.