# US 12,113,995 B2 · Neural network-based post filter for video coding > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,113,995 B2 - Document ID: US12113995B2 - Status: Granted - Kind: B2 - Application number: 17/714,024 - Filed: 2022-04-05 - Published: 2022-10-13 - Granted: 2024-10-08 - Earliest claimed: 2021-04-06 - Assignees: Lemon Inc. - Applicants: Lemon Inc. - Inventors: Yue Li; Li Zhang; Kai Zhang - Web page: https://patentgrep.com/patent/US12113995B2 ## CPC - H04N 19/436: using parallelised computational arrangements - H04N 19/124: Quantisation - H04N 19/132: Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking - H04N 19/157: Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter - H04N 19/46: Embedding additional information in the video signal during the compression process (H04N19/517, H04N19/68, H04N19/70 take precedence) - H04N 19/70: characterised by syntax aspects related to video coding, e.g. related to compression standards - H04N 19/82: involving filtering within a prediction loop - H04N 19/91: Entropy coding, e.g. variable length coding [VLC] or arithmetic coding - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A method of processing video data. The method includes determining that a supplemental enhancement information (SEI) message of a bitstream includes indicators specifying one or more neural network (NN) filter model candidates or selections for a video unit or samples within the video unit, and converting between a video media file comprising the video unit and the bitstream based on the indicators. A corresponding video coding apparatus and non-transitory computer readable medium are also disclosed. ## Claims ### Claim 1 A method of processing video data, comprising: determining that a supplemental enhancement information (SEI) message of a bitstream includes indicators specifying one or more neural network (NN) filter model candidates or selections for a video unit or samples within the video unit; and converting between a video media file comprising the video unit and the bitstream based on the indicators, wherein the indicators are included in at least one of multiple levels of the SEI message, and wherein the multiple levels comprise a picture level, a slice level, a coding tree block (CTB) level, and a coding tree unit (CTU) level. ### Claim 2 (depends on 1) The method of claim 1, wherein whether the indicators are included in the SEI message is based on whether an NN filter is enabled for the video unit. ### Claim 3 (depends on 2) The method of claim 2, wherein the indicators are excluded from the SEI message when an NN filter is disabled for the video unit. ### Claim 4 (depends on 1) The method of claim 1, wherein the indicators are coded into the bitstream with arithmetic coding or Exponential-Golomb (EG) coding. ### Claim 5 (depends on 1) The method of claim 1, wherein the indicators in the SEI message further specify one or more NN filter model indices. ### Claim 6 (depends on 1) The method of claim 1, further comprising determining that the supplemental SEI message of the bitstream includes indicators specifying one or more NN filter model indices. ### Claim 7 (depends on 6) The method of claim 6, wherein the indicators specifying the one or more NN filter model indices are coded into the bitstream with fixed length coding. ### Claim 8 (depends on 6) The method of claim 6, further comprising deriving one or more NN filter model indices or deriving on/off control for one or more NN filter models based on the SEI message. ### Claim 9 (depends on 8) The method of claim 8, wherein derivation is dependent on a parameter of a current video unit or a neighboring video unit of the current video unit. ### Claim 10 (depends on 9) The method of claim 9, wherein the parameter comprises a quantization parameter (QP) associated with the video unit or the neighboring video unit. ### Claim 11 (depends on 6) The method of claim 6, wherein the SEI message includes a first indicator in a father unit of the bitstream, and wherein the first indicator indicates how the one or more NN filter model indices are included in the SEI message for each video unit contained in the father unit or how an NN filter is used for each video unit contained in the father unit, and wherein the father unit is a set of multiple video units. ### Claim 12 (depends on 1) The method of claim 1, further comprising: determining that the SEI message of the bitstream includes a syntax element indicating whether one or more neural network (NN) filter models are enabled or disabled for the video unit. ### Claim 13 (depends on 12) The method of claim 12, wherein the syntax element is context coded or bypass coded in the bitstream. ### Claim 14 (depends on 13) The method of claim 13, wherein the syntax element is arithmetically coded using a single context or using multiple contexts. ### Claim 15 (depends on 12) The method of claim 12, wherein the multiple levels comprise a first level and a second level, and wherein whether or not to include the syntax element in the second level depends on a value of the syntax element in the first level. ### Claim 16 (depends on 1) The method of claim 1, wherein the conversion comprises encoding the video media file into the bitstream. ### Claim 17 (depends on 1) The method of claim 1, wherein the conversion includes decoding the bitstream to obtain the video media file. ### Claim 18 An apparatus for coding video data comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor cause the processor to: determine that a supplemental enhancement information (SEI) message of a bitstream includes indicators specifying one or more neural network (NN) filter model candidates or selections for a video unit or samples within the video unit; and convert between a video media file comprising the video unit and the bitstream based on the indicators, wherein the indicators are included in at least one of multiple levels of the SEI message, and wherein the multiple levels comprise a picture level, a slice level, a coding tree block (CTB) level, and a coding tree unit (CTU) level. ### Claim 19 A method for storing a bitstream of a video, comprising: determining that a supplemental enhancement information (SEI) message of a bitstream includes indicators specifying one or more neural network (NN) filter model candidates or selections for a video unit or samples within the video unit; generating the bitstream based on the SEI message; and storing the bitstream in a non-transitory computer-readable recording medium, wherein the indicators are included in at least one of multiple levels of the SEI message, and wherein the multiple levels comprise a picture level, a slice level, a coding tree block (CTB) level, and a coding tree unit (CTU) level.