Disallowing unnecessary layers in multi-layer video bitstreams

A decoding device, comprising:

• a receiver configured to receive a video bitstream including a video parameter set (VPS) and a plurality of layers, wherein no layer is neither an output layer of at least one OLS nor a direct reference layer of any other layer, wherein the VPS includes a syntax element (ols_mode_idc) and a syntax element (vps_max_layers_minus1), wherein the ols_mode_idc equal to 0 specifies that a total number of OLSs specified by the VPS is equal to the vps_max_layers_minus1+1, wherein an i-th OLS includes layers with layer indices from 0 to i, inclusive, wherein for each OLS only a highest layer in the OLS is output, wherein a layer used as a reference flag (LayerUsedAsRefLayerFlag) that indicates whether a layer is used as a reference for another layer and a layer used as an output layer flag (LayerUsedAsOutputLayerFlag) that indicates whether a layer is used as an output layer are derived based on the VPS, and wherein the LayerUsedAsRefLayerFlag[i] and the LayerUsedAsOutputLayerFlag[i] are not both zero for each value of i in a range of 0 to the syntax element (vps_max_layers_minus1), inclusive;
• a memory coupled to the receiver, the memory storing instructions;
• and
• one or more processors coupled to the memory, the one or more processors configured to execute the instructions to cause the decoding device to:
• derive a variable TotalNumOlss, which specifies the total number of OLSs specified by the VPS, based on at least the syntax element (vps_max_layers_minus1);
• and
• decode a picture from one of the plurality of layers to obtain a decoded picture.

A method of encoding implemented by a video encoder, the method comprising:

• generating a plurality of layers and a video parameter set (VPS) specifying at least one output layer set (OLS), wherein the video encoder is constrained such that no layer is neither an output layer of the at least one OLS nor a direct reference layer of any other layer, wherein the VPS includes a syntax element (ols_mode_idc) and a syntax element (vps_max_layers_minus1), wherein the ols_mode_idc equal to 0 specifies that a total number of OLSs specified by the VPS is equal to the vps_max_layers_minus1+1, wherein an i-th OLS includes layers with layer indices from 0 to i, inclusive, wherein for each OLS only a highest layer in the OLS is output, wherein the VPS further includes information configured to be used to derive a layer used as a reference flag (LayerUsedAsRefLayerFlag) that indicates whether a layer is used as a reference for another layer and a layer used as an output layer flag (LayerUsedAsOutputLayerFlag) that indicates whether a layer is used as an output layer, wherein the LayerUsedAsRefLayerFlag[i] and the LayerUsedAsOutputLayerFlag[i] are not allowed to both be zero for each value of i in a range of 0 to the syntax element (vps_max_layers_minus1), inclusive, and wherein a variable TotalNumOlss, which specifies the total number of OLSs specified by the VPS, is derived based on at least the syntax element (vps_max_layers_minus1);
• encoding the plurality of layers and the VPS into a video bitstream;
• and
• storing the video bitstream for communication toward a video decoder.