# US 12,015,765 B2 · Template matching based intra prediction > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,015,765 B2 - Document ID: US12015765B2 - Status: Granted - Kind: B2 - Application number: 17/727,570 - Filed: 2022-04-22 - Published: 2022-10-27 - Granted: 2024-06-18 - Earliest claimed: 2021-04-26 - Assignees: Tencent America LLC - Applicants: Tencent America LLC - Inventors: Ling Li; Xiang Li; Shan Liu - Web page: https://patentgrep.com/patent/US12015765B2 ## CPC - H04N 19/105: Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction - H04N 19/593: involving spatial prediction techniques - H04N 19/132: Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking - H04N 19/159: Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction - H04N 19/70: characterised by syntax aspects related to video coding, e.g. related to compression standards - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract In a method of decoding, coded information of a coding unit (CU), a template region, and a plurality of reference regions is received. The coded information includes a first syntax element indicating whether the CU is predicted based on a template matching based multiple reference line (MRL) intra prediction mode. In response to the first syntax element indicating that the CU is predicted based on the template matching based multiple reference line (MRL) intra prediction mode, a plurality of cost values is determined between (i) respective prediction samples of the template region based on samples in each of the plurality of reference regions, and (ii) reconstructed samples of the template region corresponding to the respective prediction samples. A reference region from the plurality of reference regions is determined based on the plurality of cost values. Samples of the CU are reconstructed based on the samples in the determined reference region. ## Claims ### Claim 1 A method of decoding performed in a decoder, the method comprising: receiving coded information of a coding unit (CU), a template region, and a plurality of reference regions from a coded video bitstream, the coded information including a first syntax element indicating whether the CU is predicted based on a template matching based multiple reference line (MRL) intra prediction mode and a second syntax element indicating whether the CU is intra predicted based on template based intra mode derivation (TIMD) mode including a set of candidate intra prediction modes, the template region being adjacent to the CU and each reference region of the plurality of reference regions comprising at least one reference line portion; in response to the second syntax element indicating that the CU is not intra predicted based on TIMD mode, decoding a syntax element associated with another intra coding mode, the other intra coding mode including one of matrix-based intra prediction (MIP) and most probable mode (MPM); in response to the first syntax element indicating that the CU is predicted based on the template matching based MRL intra prediction mode, determining a plurality of cost values associated with the template region and respectively corresponding to each of the plurality of reference regions, wherein each cost value is between (i) respective prediction samples of the template region based on samples in a respective one of the plurality of reference regions, and (ii) reconstructed samples of the template region corresponding to the respective prediction samples; selecting a reference region from the plurality of reference regions, the selected reference region corresponding to a lowest one among the plurality of cost values; and reconstructing samples of the CU based on the samples in the selected reference region. ### Claim 2 (depends on 1) The method of claim 1, wherein each of the plurality of cost values is determined according to a difference between (i) the respective prediction samples of the template region that is determined based on the samples in the respective one of the plurality of reference regions and (ii) the reconstructed samples of the template region corresponding to the respective prediction samples. ### Claim 3 (depends on 1) The method of claim 1, wherein the template region further comprises: one or a combination of (i) top samples in a row above a top side of the CU and (ii) side samples in a column adjacent to a left side of the CU. ### Claim 4 (depends on 3) The method of claim 3, wherein the template region further comprises: the top samples and the side samples; and an interface sample positioned above the side samples and adjacent to the top samples. ### Claim 5 (depends on 4) The method of claim 4, wherein the plurality of reference regions comprises: a first reference region including a reference line portion positioned above a top side of the template region and a column portion adjacent to a left side of the template region; a second reference region including a reference line portion above the reference line portion of the first reference region and a column portion adjacent to the column portion of the first reference region; and a third reference region including a reference line portion above the reference line portion of the second reference region and a column portion adjacent to the column portion of the second reference region. ### Claim 6 (depends on 1) The method of claim 1, wherein the template region further comprises: one or a combination of (i) top samples in a first row above a top side of the CU and top samples in a second row above the first row, and (ii) side samples in a first column adjacent to a left side of the CU and side samples in a second column along the first column. ### Claim 7 (depends on 6) The method of claim 6, wherein the template region further comprises: the top samples in the first row and the second row; the side samples in the first column and the second column; and interface samples positioned over the side samples of the first and second columns and left of the top samples of the first and second rows. ### Claim 8 (depends on 7) The method of claim 7, wherein the plurality of reference regions comprises: a first reference region including a reference line portion positioned above a top side of the template region and a column portion adjacent to a left side of the template region; and a second reference region including a reference line portion above the reference line portion of the first reference region and a column portion adjacent to the column portion of the first reference region. ### Claim 9 (depends on 1) The method of claim 1, further comprising: in response to the first syntax element indicating that the CU is predicted based on the template matching based MRL intra prediction mode, and the second syntax element indicating that the CU is intra predicted based on the TIMD mode, determining the respective prediction samples of the template region based on (i) the samples positioned in a respective one of the plurality of reference regions, and (ii) a respective candidate intra prediction mode in the set of candidate intra prediction modes; determining the cost values, according to a sum of absolute transformed differences between the respective prediction samples of the template region and the reconstructed samples of the template region corresponding to the respective prediction samples, each of the cost values corresponding to a combination of the respective one of the plurality of reference regions and an intra prediction mode; determining a combination of a reference region from the plurality of reference regions and an intra prediction mode from the set of candidate intra prediction modes that is associated with a lowest cost value among the plurality of cost values; and reconstructing the samples of the CU based on the samples in the determined combination of the reference region and the intra prediction mode. ### Claim 10 An apparatus for video decoding, comprising: processing circuitry configured to: receive coded information of a coding unit (CU), a template region, and a plurality of reference regions from a coded video bitstream, the coded information including a first syntax element indicating whether the CU is predicted based on a template matching based multiple reference line (MRL) intra prediction mode and a second syntax element indicating whether the CU is intra predicted based on template based intra mode derivation (TIMD) mode including a set of candidate intra prediction modes, the template region being adjacent to the CU and each reference region of the plurality of reference regions comprising at least one reference line portion; in response to the second syntax element indicating that the CU is not intra predicted based on TIMD mode, decode a syntax element associated with another intra coding mode, the other intra coding mode including one of matrix-based intra prediction (MIP) and most probable mode (MPM); in response to the first syntax element indicating that the CU is predicted based on the template matching based MRL intra prediction mode, determine a plurality of cost values associated with the template region and respectively corresponding to each of the plurality of reference regions, wherein each cost value is between (i) respective prediction samples of the template region based on samples in a respective one of the plurality of reference regions, and (ii) reconstructed samples of the template region corresponding to the respective prediction samples; select a reference region from the plurality of reference regions, the selected reference region corresponding to a lowest one among the plurality of cost values; and reconstruct samples of the CU based on the samples in the selected reference region. ### Claim 11 (depends on 10) The apparatus of claim 10, wherein each of the plurality of cost values is determined according to a difference between (i) the respective prediction samples of the template region that is determined based on the samples in the respective one of the plurality of reference regions and (ii) the reconstructed samples of the template region corresponding to the respective prediction samples. ### Claim 12 (depends on 10) The apparatus of claim 10, wherein the template region further comprises: one or a combination of (i) top samples in a row above a top side of the CU and (ii) side samples in a column adjacent to a left side of the CU. ### Claim 13 (depends on 12) The apparatus of claim 12, wherein the template region further comprises: the top samples and the side samples; and an interface sample positioned above the side samples and adjacent to the top samples. ### Claim 14 (depends on 13) The apparatus of claim 13, wherein the plurality of reference regions comprises: a first reference region including a reference line portion positioned above a top side of the template region and a column portion adjacent to a left side of the template region; a second reference region including a reference line portion above the reference line portion of the first reference region and a column portion adjacent to the column portion of the first reference region; and a third reference region including a reference line portion above the reference line portion of the second reference region and a column portion adjacent to the column portion of the second reference region. ### Claim 15 (depends on 10) The apparatus of claim 10, wherein the template region further comprises: one or a combination of (i) top samples in a first row above a top side of the CU and top samples in a second row above the first row, and (ii) side samples in a first column adjacent to a left side of the CU and side samples in a second column along the first column. ### Claim 16 (depends on 15) The apparatus of claim 15, wherein the template region further comprises: the top samples in the first row and the second row; the side samples in the first column and the second column; and interface samples positioned over the side samples of the first and second columns and left of the top samples of the first and second rows. ### Claim 17 (depends on 16) The apparatus of claim 16, wherein the plurality of reference regions comprises: a first reference region including a reference line portion positioned above a top side of the template region and a column portion adjacent to a left side of the template region; and a second reference region including a reference line portion above the reference line portion of the first reference region and a column portion adjacent to the column portion of the first reference region. ### Claim 18 (depends on 10) The apparatus of claim 10, wherein the processing circuitry is further configured to: in response to the first syntax element indicating that the CU is predicted based on the template matching based MRL intra prediction mode, and the second syntax element indicating that the CU is intra predicted based on the TIMD mode, determine the respective prediction samples of the template region based on (i) the samples positioned in a respective one of the plurality of reference regions, and (ii) a respective candidate intra prediction mode in the set of candidate intra prediction modes; determine the cost values, according to a sum of absolute transformed differences between the respective prediction samples of the template region and the reconstructed samples of the template region corresponding to the respective prediction samples, each of the cost values corresponding to a combination of the respective one of the plurality of reference regions and an intra prediction mode; determine a combination of a reference region from the plurality of reference regions and an intra prediction mode from the set of candidate intra prediction modes that is associated with a lowest cost value among the plurality of cost values; and reconstruct the samples of the CU based on the samples in the determined combination of the reference region and the intra prediction mode. ### Claim 19 A non-transitory computer readable storage medium storing instructions which when executed by at least one processor cause the at least one processor to perform: receiving coded information of a coding unit (CU), a template region, and a plurality of reference regions from a coded video bitstream, the coded information including a first syntax element indicating whether the CU is predicted based on a template matching based multiple reference line (MRL) intra prediction mode and a second syntax element indicating whether the CU is intra predicted based on template based intra mode derivation (TIMD) mode including a set of candidate intra prediction modes, the template region being adjacent to the CU and each reference region of the plurality of reference regions comprising at least one reference line portion; in response to the second syntax element indicating that the CU is not intra predicted based on TIMD mode, decoding a syntax element associated with another intra coding mode, the other intra coding mode including one of matrix-based intra prediction (MIP) and most probable mode (MPM); in response to the first syntax element indicating that the CU is predicted based on the template matching based MRL intra prediction mode, determining a plurality of cost values associated with the template region and respectively corresponding to each of the plurality of reference regions, wherein each cost value is between (i) respective prediction samples of the template region based on samples in a respective one of the plurality of reference regions, and (ii) reconstructed samples of the template region corresponding to the respective prediction samples; selecting a reference region from the plurality of reference regions, the selected reference region corresponding to a lowest one among the plurality of cost values; and reconstructing samples of the CU based on the samples in the selected reference region.