# US 12,192,536 B2 · Image frame error concealment method and apparatus > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,192,536 B2 - Document ID: US12192536B2 - Status: Granted - Kind: B2 - Application number: 17/616,555 - Filed: 2019-12-09 - Published: 2022-07-28 - Granted: 2025-01-07 - Earliest claimed: 2019-06-28 - Assignees: Jlq Technology Co., Ltd. - Applicants: Jlq Technology Co., Ltd. - Inventors: Hailin Qin - Web page: https://patentgrep.com/patent/US12192536B2 ## CPC - H04N 19/895: in combination with error concealment - H04N 19/105: Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction - H04N 19/109: among a plurality of temporal predictive coding modes - H04N 19/11: among a plurality of spatial predictive coding modes - H04N 19/139: Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability - H04N 19/159: Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction - H04N 19/172: the region being a picture, frame or field - H04N 19/176: the region being a block, e.g. a macroblock - H04N 19/52: by predictive encoding - H04N 19/593: involving spatial prediction techniques - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract The present disclosure discloses an image frame error concealment method including the following steps: determining whether an nth image frame has a packet loss error, and if so, not performing error concealment on the nth image frame; and performing error concealment on an (n+1)th image frame, wherein the error concealment operation is performed with reference to one or more image frames before the nth image frame, wherein n is an integer greater than or equal to 2. ## Claims ### Claim 1 An image frame error concealment method comprising the following steps: determining whether an nth image frame has a packet loss error; and if the nth image frame has a packet loss error, not performing error concealment on the nth image frame, and performing error concealment on an (n+1)th image frame, wherein the error concealment operation is performed with reference to one or more image frames before the nth image frame, wherein n is an integer greater than or equal to 2, wherein the step of performing error concealment on an (n+1)th image frame comprises: decoding a first-type intra macroblock capable of being directly and independently decoded in the (n+1)th image frame; decoding a second-type intra macroblock which depends on only the first-type intra macroblock, in the (n+1)th image frame; performing error concealment on an inter macroblock in the (n+1)th image frame with reference to one or more image frames before the nth image frame; and performing error concealment on a third-type intra macroblock not capable of being directly and independently decoded in the (n+1)th image frame. ### Claim 2 (depends on 1) The method according to claim 1, wherein the method further comprises decoding an (n+2)th image frame, the decoding operation being performed with reference to the (n+1)th image frame. ### Claim 3 (depends on 1) The method according to claim 1, wherein the step of performing error concealment on an inter macroblock in the (n+1)th image frame with reference to one or more image frames before the nth image frame comprises: decoding a motion vector of the inter macroblock; calculating candidate motion vectors; selecting a target motion vector from the candidate motion vectors; and acquiring a reference macroblock from one or more image frames before the nth image frame according to the target motion vector, and decoding the current inter macroblock according to the reference macroblock. ### Claim 4 (depends on 1) The method according to claim 1, wherein the step of performing error concealment on a third-type intra macroblock not capable of being directly and independently decoded in the (n+1)th image frame comprises: acquiring a prediction mode of the current third-type intra macroblock; searching remaining third-type intra macroblocks for a related third-type intra macroblock which takes the current third-type intra macroblock as a reference macroblock and has the same prediction mode; acquiring a plurality of sets of candidate reference pixels of the current third-type intra macroblock and the related third-type intra macroblock; decoding the current third-type intra macroblock and the related third-type intra macroblock by using the plurality of sets of candidate reference pixels, to select a target reference pixel; and decoding the current third-type intra macroblock and the related third-type intra macroblock according to the target reference pixel. ### Claim 5 (depends on 1) A decoding apparatus, comprising: a memory for storing instructions executable by a processor; and the processor for executing the instructions to implement the method according to claim 1. ### Claim 6 (depends on 1) A non-transitory computer-readable storage medium having computer program codes stored thereon which, when executed by a processor, implement the method according to claim 1. ### Claim 7 An image frame error concealment method comprising the following steps: determining whether an nth image frame has a packet loss error; and if the nth image frame has a packet loss error, not performing error concealment on the nth image frame, and performing error concealment on an (n+1)th image frame, wherein the error concealment operation is performed with reference to one or more image frames before the nth image frame, wherein n is an integer greater than or equal to 2, wherein when it is determined that the nth image frame has a packet loss error, the method further comprises: attempting to decode the nth image frame; and determining whether one or more macroblocks can be obtained by the decoding, and if so, the error concealment operation comprising performing error concealment with reference to the one or more macroblocks and one or more image frames before the nth image frame, and if not, discarding the nth image frame; wherein the nth image frame is not displayed. ### Claim 8 (depends on 7) The method according to claim 7, wherein the step of performing error concealment on an (n+1)th image frame comprises: decoding a first-type intra macroblock capable of being directly and independently decoded in the (n+1)th image frame; decoding a second-type intra macroblock which depends on only the first-type intra macroblock capable of being directly and independently decoded, in the (n+1)th image frame; performing error concealment on an inter macroblock in the (n+1)th image frame with reference to one or more macroblocks of the nth image frame and one or more image frames before the nth image frame; and performing error concealment on a third-type intra macroblock not capable of being directly and independently decoded in the (n+1)th image frame. ### Claim 9 (depends on 8) The method according to claim 8, wherein the step of performing error concealment on an inter macroblock in the (n+1)th image frame with reference to one or more image frames before the nth image frame comprises: decoding a motion vector of the inter macroblock; calculating candidate motion vectors; determining a position of an image frame in which a reference macroblock is located according to the motion vector; when the reference macroblock is located in the nth image frame, directly decoding the inter macroblock according to the reference macroblock, and ending the process; when the reference macroblock is located in one or more image frames before the nth image frame, selecting a target motion vector from the candidate motion vectors; and acquiring a reference macroblock from one or more image frames before the nth image frame according to the target motion vector, and decoding the current inter macroblock according to the reference macroblock. ### Claim 10 (depends on 8) The method according to claim 8, wherein the step of performing error concealment on a third-type intra macroblock not capable of being directly and independently decoded in the (n+1)th image frame comprises: acquiring a prediction mode of the current third-type intra macroblock; searching remaining third-type intra macroblocks for a related third-type intra macroblock which takes the current third-type intra macroblock as a reference macroblock and has the same prediction mode; acquiring a plurality of sets of candidate reference pixels of the current third-type intra macroblock and the related third-type intra macroblock; decoding the current third-type intra macroblock and the related third-type intra macroblock by using the plurality of sets of candidate reference pixels, to select a target reference pixel; and decoding the current third-type intra macroblock and the related third-type intra macroblock according to the target reference pixel. ### Claim 11 An image frame error concealment apparatus, comprising: an error detection module for determining whether an nth image frame has a packet loss error; and an error concealment module for when the nth image frame has a packet loss error, not restoring the nth image frame, and performing error concealment on an (n+1)th image frame, wherein the error concealment operation is performed with reference to one or more image frames before the nth image frame, wherein n is an integer greater than or equal to 2, wherein the error concealment module comprises: a first decoding unit for decoding a first-type intra macroblock capable of being directly and independently decoded in the (n+1)th image frame; a second decoding unit for decoding a second-type intra macroblock which depends on only the first-type intra macroblock capable of being directly and independently decoded, in the (n+1)th image frame; a first error concealment unit for performing error concealment on an inter macroblock in the (n+1)th image frame with reference to one or more image frames before the nth image frame; and a second error concealment unit for performing error concealment on a third-type intra macroblock not capable of being directly and independently decoded in the (n+1)th image frame. ### Claim 12 (depends on 11) The apparatus according to claim 11, wherein the error concealment module is further used to decode an (n+2)th image frame, the decoding operation being performed with reference to the (n+1)th image frame. ### Claim 13 (depends on 11) The apparatus according to claim 11, wherein the first error concealment unit is configured to perform error concealment as follows: decoding a motion vector of the inter macroblock; calculating candidate motion vectors; selecting a target motion vector from the candidate motion vectors; and acquiring a reference macroblock from one or more image frames before the nth image frame according to the target motion vector, and decoding the current inter macroblock according to the reference macroblock. ### Claim 14 (depends on 11) The apparatus according to claim 11, wherein the second error concealment unit is configured to perform error concealment as follows: acquiring a prediction mode of the current third-type intra macroblock; searching remaining third-type intra macroblocks for a related third-type intra macroblock which takes the current third-type intra macroblock as a reference macroblock and has the same prediction mode; acquiring a plurality of sets of candidate reference pixels of the current third-type intra macroblock and the related third-type intra macroblock; decoding the current third-type intra macroblock and the related third-type intra macroblock by using the plurality of sets of candidate reference pixels, to select a target reference pixel; and decoding the current third-type intra macroblock and the related third-type intra macroblock according to the target reference pixel. ### Claim 15 An image frame error concealment apparatus, comprising: an error detection module for determining whether an nth image frame has a packet loss error; and an error concealment module for when the nth image frame has a packet loss error, not restoring the nth image frame, and performing error concealment on an (n+1)th image frame, wherein the error concealment operation is performed with reference to one or more image frames before the nth image frame, wherein n is an integer greater than or equal to 2, wherein the error concealment module is further used to attempt to decode the nth image frame and determine whether one or more macroblocks can be obtained by the decoding when it is determined that the nth image frame has a packet loss error; wherein if one or more macroblocks can be obtained by the decoding, the error concealment module is configured to perform error concealment with reference to the one or more macroblocks and one or more image frames before the nth image frame, and if one or more macroblocks cannot be obtained by the decoding, the error concealment module is configured to discard the nth image frame, and wherein the nth image frame is not displayed. ### Claim 16 (depends on 15) The apparatus according to claim 15, wherein the error concealment module comprises: a first decoding unit for decoding a first-type intra macroblock capable of being directly and independently decoded in the (n+1)th image frame; a second decoding unit for decoding a second-type intra macroblock which depends on only the first-type intra macroblock capable of being directly and independently decoded, in the (n+1)th image frame; a first error concealment unit for performing error concealment on an inter macroblock in the (n+1)th image frame with reference to one or more macroblocks of the nth image frame and one or more image frames before the nth image frame; and a second error concealment unit for performing error concealment on a third-type intra macroblock not capable of being directly and independently decoded in the (n+1)th image frame. ### Claim 17 (depends on 16) The apparatus according to claim 16, wherein the first error concealment unit is configured to perform error concealment as follows: decoding a motion vector of the inter macroblock; calculating candidate motion vectors; determining a position of an image frame in which a reference macroblock is located according to the motion vector; when the reference macroblock is located in the nth image frame, directly decoding the inter macroblock according to the reference macroblock, and ending the process; when the reference macroblock is located in one or more image frames before the nth image frame, selecting a target motion vector from the candidate motion vectors; and acquiring a reference macroblock from one or more image frames before the nth image frame according to the target motion vector, and decoding the current inter macroblock according to the reference macroblock. ### Claim 18 (depends on 16) The apparatus according to claim 16, wherein the second error concealment unit is configured to perform error concealment as follows: acquiring a prediction mode of the current third-type intra macroblock; searching remaining third-type intra macroblocks for a related third-type intra macroblock which takes the current third-type intra macroblock as a reference macroblock and has the same prediction mode; acquiring a plurality of sets of candidate reference pixels of the current third-type intra macroblock and the related third-type intra macroblock; decoding the current third-type intra macroblock and the related third-type intra macroblock by using the plurality of sets of candidate reference pixels, to select a target reference pixel; and decoding the current third-type intra macroblock and the related third-type intra macroblock according to the target reference pixel.