# US 11,909,996 B2 · Ordering of NAL units in coded video > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,909,996 B2 - Document ID: US11909996B2 - Status: Granted - Kind: B2 - Application number: 17/989,399 - Filed: 2022-11-17 - Published: 2023-03-16 - Granted: 2024-02-20 - Earliest claimed: 2020-05-22 - Assignees: Bytedance Inc. - Applicants: Bytedance Inc. - Inventors: Ye-kui Wang - Web page: https://patentgrep.com/patent/US11909996B2 ## CPC - H04N 19/46: Embedding additional information in the video signal during the compression process (H04N19/517, H04N19/68, H04N19/70 take precedence) - H04N 7/01: Conversion of standards {, e.g. involving analogue television standards or digital television standards processed at pixel level} - H04N 19/172: the region being a picture, frame or field - H04N 19/188: {the unit being a video data packet, e.g. a network abstraction layer [NAL] unit} - H04N 19/70: characterised by syntax aspects related to video coding, e.g. related to compression standards - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract Examples of video encoding methods and apparatus and video decoding methods and apparatus are described. An example method of video processing includes performing a conversion between a video and a bitstream of the video. The bitstream includes one or more layers including one or more network abstraction layer (NAL) units according to a format rule that specifies that a first layer identifier in a header of an end of sequence network abstraction layer (EOS NAL) unit is required to be equal to a second layer identifier of one of the one or more layers in the bitstream. ## Claims ### Claim 1 A method of video processing, comprising: performing a conversion between a video and a bitstream of the video according to a first format rule and a third format rule, wherein the bitstream comprises one or more layers comprising one or more network abstraction layer (NAL) units, wherein the first format rule specifies that a first layer identifier of an end of sequence network abstraction layer (EOS NAL) unit is required to be equal to a second layer identifier of one of the one or more layers in the bitstream, and wherein the third format rule specifies that a picture unit (PU) of a particular layer after an EOS NAL unit of the particular layer shall be an intra random access point (IRAP) PU or a gradual decoding refresh (GDR) PU. ### Claim 2 (depends on 1) The method of claim 1, wherein the first format rule further allows for inclusion of more than one EOS NAL units in a PU. ### Claim 3 (depends on 1) The method of claim 1, wherein the first format rule specifies that the first layer identifier of the EOS NAL unit is required to be less than or equal to a third layer identifier of a video coding layer (VCL) NAL unit associated with the EOS NAL unit. ### Claim 4 (depends on 1) The method of claim 1, wherein the conversion is performed according to a second format rule, wherein the bitstream comprises one or more layers comprising one or more PUs comprising one or more NAL units, and wherein the second format rule specifies that, responsive to a first NAL unit indicating an end of sequence in a PU, the first NAL unit is a last NAL unit among all NAL units within the PU other than another NAL unit, if present, indicating another end of sequence or indicating an end of the bitstream, if present. ### Claim 5 (depends on 4) The method of claim 4, wherein the another NAL unit is an EOS NAL unit. ### Claim 6 (depends on 4) The method of claim 4, wherein the another NAL unit is an end of bitstream (EOB) NAL unit. ### Claim 7 (depends on 4) The method of claim 4, wherein the PU comprises a picture header NAL unit, a coded picture comprising one or more video coding layer NAL units and zero or more non-video coding layer NAL units. ### Claim 8 (depends on 1) The method of claim 1, wherein the performing the conversion comprises encoding the video into the bitstream. ### Claim 9 (depends on 1) The method of claim 1, wherein the performing the conversion comprises decoding the video from the bitstream. ### Claim 10 (depends on 1) The method of claim 1, wherein the conversion is performed according to a fourth format rule, wherein the fourth format rule specifies that, when present, an EOS raw byte sequence payload (RBSP) syntax structure specifies that a next subsequent PU that belongs to a same layer as the EOS NAL unit in the bitstream in a decoding order is of a particular PU type from an IRAP PU type or a GDR PU type. ### Claim 11 An apparatus for processing video data comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to: perform a conversion between a video and a bitstream of the video according to a first format rule and a third format rule, wherein the bitstream comprises one or more layers comprising one or more network abstraction layer (NAL) units, wherein the first format rule specifies that a first layer identifier of an end of sequence network abstraction layer (EOS NAL) unit is required to be equal to a second layer identifier of one of the one or more layers in the bitstream, and wherein the third format rule specifies that a picture unit (PU) of a particular layer after an EOS NAL unit of the particular layer shall be an intra random access point (IRAP) PU or a gradual decoding refresh (GDR) PU. ### Claim 12 (depends on 11) The apparatus of claim 11, wherein the first format rule further allows for inclusion of more than one EOS NAL units in a PU, and wherein the first format rule specifies that the first layer identifier of the EOS NAL unit is required to be less than or equal to a third layer identifier of a video coding layer (VCL) NAL unit associated with the EOS NAL unit. ### Claim 13 (depends on 11) The apparatus of claim 11, wherein the conversion is performed according to a second format rule, wherein the bitstream comprises one or more layers comprising one or more PUs comprising one or more NAL units, and wherein the second format rule specifies that, responsive to a first NAL unit indicating an end of sequence in a PU, the first NAL unit is a last NAL unit among all NAL units within the PU other than another NAL unit, if present, indicating another end of sequence or indicating an end of the bitstream, if present. ### Claim 14 (depends on 13) The apparatus of claim 13, wherein the another NAL unit is an EOS NAL unit. ### Claim 15 (depends on 13) The apparatus of claim 13, wherein the another NAL unit is an end of bitstream (EOB) NAL unit. ### Claim 16 (depends on 13) The apparatus of claim 13, wherein the PU comprises a picture header NAL unit, a coded picture comprising one or more video coding layer NAL units and zero or more non-video coding layer NAL units. ### Claim 17 A non-transitory computer-readable storage medium storing instructions that cause a processor to: perform a conversion between a video and a bitstream of the video according to a first format rule and a third format rule, wherein the bitstream comprises one or more layers comprising one or more network abstraction layer (NAL) units, wherein the first format rule specifies that a first layer identifier of an end of sequence network abstraction layer (EOS NAL) unit is required to be equal to a second layer identifier of one of the one or more layers in the bitstream, and wherein the third format rule specifies that a picture unit (PU) of a particular layer after an EOS NAL unit of the particular layer shall be an intra random access point (IRAP) PU or a gradual decoding refresh (GDR) PU. ### Claim 18 (depends on 17) The non-transitory computer-readable storage medium of claim 17, wherein the conversion is performed according to a second format rule, wherein the bitstream comprises one or more layers comprising one or more PUs comprising one or more NAL units, and wherein the second format rule specifies that, responsive to a first NAL unit indicating an end of sequence in a PU, the first NAL unit is a last NAL unit among all NAL units within the PU other than another NAL unit, if present, indicating another end of sequence or indicating an end of the bitstream, if present. ### Claim 19 (depends on 18) The non-transitory computer-readable storage medium of claim 18, wherein the another NAL unit is an EOS NAL unit or an end of bitstream (EOB) NAL unit. ### Claim 20 A non-transitory computer-readable recording medium storing a bitstream of a video which is generated by a method performed by a video processing apparatus, wherein the method comprises: generating the bitstream of the video according to a first format rule and a third format rule, wherein the bitstream comprises one or more layers comprising one or more network abstraction layer (NAL) units, wherein the first format rule specifies that a first layer identifier of an end of sequence network abstraction layer (EOS NAL) unit is required to be equal to a second layer identifier of one of the one or more layers in the bitstream, wherein the third format rule specifies that a picture unit (PU) of a particular layer after an EOS NAL unit of the particular layer shall be an intra random access point (IRAP) PU or a gradual decoding refresh (GDR) PU.