# US 12,192,493 B2 · Techniques for generation of a conformant output sub-bitstream

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,192,493 B2
- Document ID: US12192493B2
- Status: Granted
- Kind: B2
- Application number: 18/488,692
- Filed: 2023-10-17
- Published: 2024-02-08
- Granted: 2025-01-07
- Earliest claimed: 2020-05-22
- Assignees: Bytedance Inc.
- Applicants: Bytedance Inc.
- Inventors: Ye-kui Wang
- Web page: https://patentgrep.com/patent/US12192493B2

## CPC

- H04N 19/188: {the unit being a video data packet, e.g. a network abstraction layer [NAL] unit}
- H04N 21/234381: {by altering the temporal resolution, e.g. decreasing the frame rate by frame skipping}
- H04N 19/172: the region being a picture, frame or field
- H04N 19/30: using hierarchical techniques, e.g. scalability (H04N19/63 takes precedence)
- H04N 19/31: in the temporal domain
- H04N 19/70: characterised by syntax aspects related to video coding, e.g. related to compression standards
- H04N 21/234327: {by decomposing into layers, e.g. base layer and one or more enhancement layers}
- H04N 21/2662: Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
- H04N 21/4345: {Extraction or processing of SI, e.g. extracting service information from an MPEG stream}
- H04N 21/4347: {Demultiplexing of several video streams}
- H04N 21/440281: {by altering the temporal resolution, e.g. by frame skipping (television signal recording using magnetic recording on tape for reproducing at a rate different from the recording rate H04N5/783)}
- H04N 21/8451: {using Advanced Video Coding [AVC]}
- H04N 19/176: the region being a block, e.g. a macroblock
- H04N 19/61: in combination with predictive coding
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Examples of video encoding methods and apparatus and video decoding methods and apparatus are described. An example method of video processing includes performing a conversion between a video including multiple layers and a bitstream of the video according to a rule, wherein the rule specifies that, in a first process of sub-bitstream extraction to output a first output sub-bitstream, the first output sub-bitstream is extracted without removing network abstraction layer (NAL) units of a particular type and having a particular NAL unit header identifier value, and wherein the particular type includes an access unit delimiter (AUD) NAL unit.

## Claims

### Claim 1

A method of processing video data, comprising:
performing a conversion between a video including one or more layers and a bitstream of the video according to a rule,
wherein the rule specifies that, in a first process of sub-bitstream extraction to output a first output sub-bitstream, the first output sub-bitstream is extracted without removing network abstraction layer (NAL) units of a particular type and having a particular NAL unit header identifier value,
wherein the particular type includes an access unit delimiter (AUD) NAL unit, and
wherein the rule further specifies, in a second process of sub-bitstream extraction to output a second output sub-bitstream, whether to generate non-scalable-nested supplementary enhancement information (SEI) messages by extracting scalable-nested SEI messages from a scalable nesting SEI message is based on a first flag indicating whether the scalable nesting SEI message applies to specific output layer sets (OLSs) and a second flag indicating whether the scalable nesting SEI message applies to all subpictures or only to specific subpictures, and wherein the rule further specifies to generate the non-scalable-nested SEI message when a set of conditions are satisfied, wherein the set of conditions comprises that the second flag has a second value that specifies that the SEI message that applies to the specific OLSs applies to all subpictures of the specified OLSs.

### Claim 2 (depends on 1)

The method of claim 1, wherein the particular type includes at least one of the following: a video parameter set (VPS) NAL unit, a decoding capability information NAL unit, an end of bitstream NAL unit.

### Claim 3 (depends on 1)

The method of claim 1, wherein the particular type includes a SEI NAL unit containing the non-scalable nested SEI message with a payload type that is equal to 0, 1, 130 or 203.

### Claim 4 (depends on 1)

The method of claim 1, wherein the particular NAL unit header identifier value includes a layer identifier value that is included in a list of layer values for the first output sub-bitstream.

### Claim 5 (depends on 1)

The method of claim 1, wherein the rule further specifies that, a NAL unit of the particular type is not removed from the first output sub-bitstream regardless of a NAL unit header identifier value associated with the NAL unit of the particular type.

### Claim 6 (depends on 1)

The method of claim 1, wherein the rule further specifies that, when an SEI NAL unit contains a scalable-nested SEI message with a payload type equal to 0 (buffering period (BP)), 1 (picture timing (PT)), 130 (decoding unit information (DUI)), or 203 (subpicture level information (SLI)), the SEI NAL unit excludes any other SEI message with a payload type not equal to 0 (BP), 1 (PT), 130 (DUI), 203 (SLI), and 133 (scalable nesting).

### Claim 7 (depends on 1)

The method of claim 1, wherein the rule further specifies that, in a third process of sub-bitstream extraction to output a third output sub-bitstream, a maximum allowed value of a temporal layer identifier of the third output sub-bitstream is in a range of zero to a value based on a first syntax element in a video parameter set referenced by the bitstream, and
wherein the first syntax element specifies a maximum number of temporal sublayers that are allowed to be present in a layer specified by the video parameter set minus 1.

### Claim 8 (depends on 1)

The method of claim 1, wherein the rule further specifies, in the first process of sub-bitstream extraction, to remove NAL units that include the scalable nesting SEI message applied to layers that are not included in a target output layer set (OLS),
wherein the scalable nesting SEI message is associated with a flag having a value equal to a particular value that specifies the scalable nesting SEI message applies to the layers, and
wherein the rule further specifies to remove the NAL units that include the SEI message having no value in a first list (NestingLayerID [i]) that is in a second list (LayerIdInOls[targetOlsIdx]), wherein the first list specifies NAL unit header identifier values of layers to which the scalable nesting SEI message applies and i is in a range of 0 to NumLayers−1, NumLayers indicates a number of the layers to which the scalable nesting SEI message applies, and the second list specifies the NAL unit header identifier values in the target output layer set with a target output layer index.

### Claim 9 (depends on 8)

The method of claim 8, wherein the particular value is 0.

### Claim 10 (depends on 1)

The method of claim 1, wherein the conversion includes encoding the video into the bitstream.

### Claim 11 (depends on 1)

The method of claim 1, wherein the conversion includes decoding the video from the bitstream.

### Claim 12

An apparatus for processing video data comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to:
perform a conversion between a video including one or more layers and a bitstream of the video according to a rule,
wherein the rule specifies that, in a first process of sub-bitstream extraction to output a first output sub-bitstream, the first output sub-bitstream is extracted without removing network abstraction layer (NAL) units of a particular type and having a particular NAL unit header identifier value,
wherein the particular type includes an access unit delimiter (AUD) NAL unit,
wherein the rule further specifies, in a second process of sub-bitstream extraction to output a second output sub-bitstream, whether to generate non-scalable-nested supplementary enhancement information (SEI) messages by extracting scalable-nested SEI messages from a scalable nesting SEI message is based on a first flag indicating whether the scalable nesting SEI message applies to specific output layer sets (OLSs) and a second flag indicating whether the scalable nesting SEI message applies to all subpictures or only to specific subpictures, and wherein the rule further specifies to generate the non-scalable-nested SEI message when a set of conditions are satisfied, wherein the set of conditions comprises that the second flag has a second value that specifies that the SEI message that applies to the specific OLSs applies to all subpictures of the specified OLSs.

### Claim 13 (depends on 12)

The apparatus of claim 12, wherein the particular type includes at least one of the following: a video parameter set (VPS) NAL unit, a decoding capability information NAL unit, an end of bitstream NAL unit.

### Claim 14 (depends on 12)

The apparatus of claim 12, wherein the particular type includes a SEI NAL unit containing the non-scalable nested SEI message with a payload type that is equal to 0, 1, 130, or 203.

### Claim 15 (depends on 12)

The apparatus of claim 12, wherein the particular NAL unit header identifier value includes a layer identifier value that is included in a list of layer values for the first output sub-bitstream.

### Claim 16

A non-transitory computer-readable storage medium storing instructions that cause a processor to:
perform a conversion between a video including one or more layers and a bitstream of the video according to a rule,
wherein the rule specifies that, in a first process of sub-bitstream extraction to output a first output sub-bitstream, the first output sub-bitstream is extracted without removing network abstraction layer (NAL) units of a particular type and having a particular NAL unit header identifier value,
wherein the particular type includes an access unit delimiter (AUD) NAL unit,
wherein the rule further specifies, in a second process of sub-bitstream extraction to output a second output sub-bitstream, whether to generate non-scalable-nested supplementary enhancement information (SEI) messages by extracting scalable-nested SEI messages from a scalable nesting SEI message is based on a first flag indicating whether the scalable nesting SEI message applies to specific output layer sets (OLSs) and a second flag indicating whether the scalable nesting SEI message applies to all subpictures or only to specific subpictures, and wherein the rule further specifies to generate the non-scalable-nested SEI message when a set of conditions are satisfied, wherein the set of conditions comprises that the second flag has a second value that specifies that the SEI message that applies to the specific OLSs applies to all subpictures of the specified OLSs.

### Claim 17 (depends on 16)

The non-transitory computer-readable storage medium of claim 16, wherein the particular type includes at least one of the following: a video parameter set (VPS) NAL unit, a decoding capability information NAL unit, an end of bitstream NAL unit.

### Claim 18

A non-transitory computer-readable recording medium storing a bitstream of a video which is generated by a method performed by a video processing apparatus, wherein the method comprises:
generating the bitstream of the video including one or more layers according to a rule,
wherein the rule specifies that, in a first process of sub-bitstream extraction to output a first output sub-bitstream, the first output sub-bitstream is extracted without removing network abstraction layer (NAL) units of a particular type and having a particular NAL unit header identifier value,
wherein the particular type includes an access unit delimiter (AUD) NAL unit, and
wherein the rule further specifies, in a second process of sub-bitstream extraction to output a second output sub-bitstream, whether to generate non-scalable-nested supplementary enhancement information (SEI) messages by extracting scalable-nested SEI messages from a scalable nesting SEI message is based on a first flag indicating whether the scalable nesting SEI message applies to specific output layer sets (OLSs) and a second flag indicating whether the scalable nesting SEI message applies to all subpictures or only to specific subpictures, and wherein the rule further specifies to generate the non-scalable-nested SEI message when a set of conditions are satisfied, wherein the set of conditions comprises that the second flag has a second value that specifies that the SEI message that applies to the specific OLSs applies to all subpictures of the specified OLSs.

### Claim 19 (depends on 18)

The non-transitory computer-readable recording medium of claim 18, wherein the particular type includes at least one of the following: a video parameter set (VPS) NAL unit, a decoding capability information NAL unit, an end of bitstream NAL unit.

### Claim 20 (depends on 18)

The non-transitory computer-readable recording medium of claim 18, wherein the particular type includes a SEI NAL unit containing the non-scalable nested SEI message with a payload type that is equal to 0, 1, 130, or 203.
