# US 12,192,490 B2 · Pictures with mixed NAL unit types

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,192,490 B2
- Document ID: US12192490B2
- Status: Granted
- Kind: B2
- Application number: 17/471,066
- Filed: 2021-09-09
- Published: 2021-12-30
- Granted: 2025-01-07
- Earliest claimed: 2019-03-11
- Assignees: Huawei Technologies Co., Ltd.
- Applicants: Huawei Technologies Co., Ltd.
- Inventors: Ye-Kui Wang; Fnu Hendry
- Web page: https://patentgrep.com/patent/US12192490B2

## CPC

- H04N 19/188: {the unit being a video data packet, e.g. a network abstraction layer [NAL] unit}
- H04N 19/597: specially adapted for multi-view video sequence encoding
- H04N 19/105: Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
- H04N 19/159: Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
- H04N 19/172: the region being a picture, frame or field
- H04N 19/196: being specially adapted for the computation of encoding parameters, e.g. by averaging previously computed encoding parameters (processing of motion vectors H04N19/513)
- H04N 19/433: characterised by techniques for memory access
- H04N 19/44: Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
- H04N 19/70: characterised by syntax aspects related to video coding, e.g. related to compression standards
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A video coding mechanism is disclosed. The mechanism includes receiving a bitstream comprising a plurality of sub-pictures associated with a picture and a flag. The sub-pictures are contained in a plurality of video coding layer (VCL) network abstraction layer (NAL) units. A NAL unit type value is the same for all VCL NAL units associated with the picture when the flag is set to a first value. When the flag is set to a second value, a first NAL unit type value for VCL NAL units containing one or more of the sub-pictures of the picture is different than a second NAL unit type value for VCL NAL units containing one or more of the sub-pictures of the picture. The sub-pictures are decoded based on the NAL unit type values. The sub-pictures are forwarded for display as part of a decoded video sequence.

## Claims

### Claim 1

A method implemented in a decoder, the method comprising:
receiving a bitstream comprising a picture and a flag, wherein the picture is associated with a plurality of video coding layer (VCL) network abstraction layer (NAL) units, and wherein the flag is a mixed_nalu_types_in_pic_flag;
determining a NAL unit type value is the same for all of the VCL NAL units of the picture when the flag is set to a first value;
determining the VCL NAL units of the picture have a plurality of different NAL unit type values when the flag is set to a second value; and
decoding the picture based on the flag.

### Claim 2 (depends on 1)

The method of claim 1, wherein the bitstream includes a picture parameter set (PPS) including the flag.

### Claim 3 (depends on 1)

The method of claim 1, wherein the different NAL unit type values indicate the picture contains an intra-random access point (IRAP) VCL NAL unit and a non-IRAP VCL NAL unit.

### Claim 4 (depends on 1)

The method of claim 1, wherein the different NAL unit type values include a first NAL unit type value equal to Instantaneous Decoding Refresh (IDR) with random access decodable leading picture (IDR_W_RADL), IDR with no leading pictures (IDR_N_LP), or clean random access (CRA) NAL unit type (CRA_NUT).

### Claim 5 (depends on 4)

The method of claim 4, wherein the different NAL unit type values include a second NAL unit type value equal to trailing picture NAL unit type (TRAIL_NUT).

### Claim 6 (depends on 2)

The method of claim 2, wherein the mixed_nalu_types_in_pic_flag is equal to one when specifying that each picture referring to the PPS has more than one VCL NAL unit and the VCL NAL units do not have the same value of NAL unit type (nal_unit_type), and wherein mixed_nalu_types_in_pic_flag is equal to zero when specifying that each picture referring to the PPS has one or more VCL NAL units and the VCL NAL units have the same value of nal_unit_type.

### Claim 7

A method implemented in an encoder, the method comprising:
determining whether a picture contains a plurality of sub-pictures of different types;
encoding the sub-pictures of the picture into a plurality of video coding layer (VCL) network abstraction layer (NAL) units in a bitstream;
encoding into the bitstream a flag set to a first value when a first NAL unit type value is the same for all VCL NAL units associated with the picture and set to a second value when the first NAL unit type value for VCL NAL units containing one or more of the sub-pictures of the picture is different than a second NAL unit type value for VCL NAL units containing one or more of the sub-pictures of the picture, wherein the flag is a mixed_nalu_types_in_pic_flag; and
storing the bitstream for communication toward a decoder.

### Claim 8 (depends on 7)

The method of claim 7, further comprising encoding a picture parameter set (PPS) into the bitstream, wherein the flag is encoded into the PPS.

### Claim 9 (depends on 7)

The method of claim 7, wherein the first NAL unit type value indicates the picture contains an intra-random access point (IRAP) sub-picture, and wherein the second NAL unit type value indicates the picture contains a non-IRAP sub-picture.

### Claim 10 (depends on 7)

The method of claim 7, wherein the first NAL unit type value is equal to Instantaneous Decoding Refresh (IDR) with random access decodable leading picture (IDR_W_RADL), IDR with no leading pictures (IDR_N_LP), or clean random access (CRA) NAL unit type (CRA_NUT).

### Claim 11 (depends on 7)

The method of claim 7, wherein the second NAL unit type value is equal to trailing picture NAL unit type (TRAIL_NUT), random access decodable leading picture NAL unit type (RADL_NUT), or random access skipped leading picture (RASL) NAL unit type (RASL_NUT).

### Claim 12 (depends on 8)

The method of claim 8, wherein the mixed_nalu_types_in_pic_flag is equal to one when specifying that the picture referring to the PPS has more than one of the VCL NAL units and the VCL NAL units do not have the same value of NAL unit type (nal_unit_type), and wherein mixed_nalu_types_in_pic_flag is equal to zero when specifying the picture referring to the PPS has one or more of the VCL NAL units and the VCL NAL units have the same value of nal_unit_type.

### Claim 13

A decoder comprising:
a receiver configured to receive a bitstream comprising a picture and a flag, wherein the picture is associated with a plurality of video coding layer (VCL) network abstraction layer (NAL) units, wherein the flag is a mixed_nalu_types_in_pic_flag; and
a processor coupled to the receiver and configured to:
determine a NAL unit type value is the same for all of the VCL NAL units of the picture when the flag is set to a first value;
determine the VCL NAL units of the picture have a plurality of different NAL unit type values when the flag is set to a second value; and
decode the picture based on the flag.

### Claim 14 (depends on 13)

The decoder of claim 13, wherein the bitstream includes a picture parameter set (PPS) including the flag.

### Claim 15 (depends on 13)

The decoder of claim 13, wherein the different NAL unit type values include a first NAL unit type value equal to Instantaneous Decoding Refresh (IDR) with random access decodable leading picture (IDR_W_RADL), IDR with no leading pictures (IDR_N_LP), or clean random access (CRA) NAL unit type (CRA_NUT).

### Claim 16 (depends on 15)

The decoder of claim 15, wherein the different NAL unit type values include a second NAL unit type value equal to trailing picture NAL unit type (TRAIL NUT).

### Claim 17 (depends on 14)

The decoder of claim 14, wherein the mixed_nalu_types_in_pic_flag is equal to one when specifying that each picture referring to the PPS has more than one VCL NAL unit and the VCL NAL units do not have the same value of NAL unit type (nal_unit_type), and wherein mixed_nalu_types_in_pic_flag is equal to zero when specifying that each picture referring to the PPS has one or more VCL NAL units and the VCL NAL units have the same value of nal_unit_type.

### Claim 18

A method implemented in a decoder, the method comprising:
receiving, by a receiver of the decoder, a bitstream comprising a plurality of sub-pictures associated with a picture and a flag, wherein the sub-pictures are contained in a plurality of video coding layer (VCL) network abstraction layer (NAL) units, wherein the flag is a mixed_nalu_types_in_pic_flag;
determining, by a processor of the decoder, that a first NAL unit type value that is the same for all of the VCL NAL units associated with the picture when the flag is set to a first value;
determining, by the processor, that a first NAL unit type value for a first set of the VCL NAL units containing one or more of the sub-pictures of the picture is different than a second NAL unit type value for a second set of the VCL NAL units containing one or more of the sub-pictures of the picture when the flag is set to a second value; and
decoding, by the processor, one or more of the sub-pictures based on the first NAL unit type value or the second NAL unit type value.

### Claim 19 (depends on 18)

The method of claim 18, wherein the bitstream includes a picture parameter set (PPS) including the flag.

### Claim 20 (depends on 18)

The method of claim 18, wherein the first NAL unit type value is equal to
Instantaneous Decoding Refresh (IDR) with random access decodable leading picture (IDR_W_RADL), IDR with no leading pictures (IDR_N_LP), or clean random access (CRA) NAL unit type (CRA_NUT).

### Claim 21 (depends on 18)

The method of claim 18, wherein the different NAL unit type values include a second NAL unit type value equal to trailing picture NAL unit type (TRAIL_NUT).

### Claim 22 (depends on 19)

The method of claim 19, wherein the mixed_nalu_types_in_pic_flag is equal to one when specifying that each picture referring to the PPS has more than one VCL NAL unit and the VCL NAL units do not have the same value of NAL unit type (nal_unit_type), and wherein mixed_nalu_types_in_pic_flag is equal to zero when specifying that each picture referring to the PPS has one or more VCL NAL units and the VCL NAL units have the same value of nal_unit_type.