# US 2019/0320192 A1 · Apparatus, a method and a computer program for video coding and decoding > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0320192 A1 - Document ID: US20190320192A1 - Status: Published application - Kind: A1 - Application number: 16/450,464 - Filed: 2019-06-24 - Published: 2019-10-17 - Earliest claimed: 2014-03-17 - Assignees: Nokia Technologies Oy - Applicants: Nokia Technologies Oy - Inventors: Miska Matias HANNUKSELA - Web page: https://patentgrep.com/patent/US20190320192A1 ## CPC - H04N 19/31: in the temporal domain - G06F 9/268: {Microinstruction selection not based on processing results, e.g. interrupt, patch, first cycle store, diagnostic programs} - G06F 9/328: {for runtime instruction patching} - G06F 9/3802: {Instruction prefetching} - G06F 12/0638: {Combination of memories, e.g. ROM and RAM such as to permit replacement or supplementing of words in one module by words in another module (address formation of the next microinstruction G06F9/26; masking faults in memories by using spares or by reconfiguring G11C29/70)} - H04N 19/172: the region being a picture, frame or field - H04N 19/187: the unit being a scalable video layer - H04N 19/433: characterised by techniques for memory access - H04N 19/46: Embedding additional information in the video signal during the compression process (H04N19/517, H04N19/68, H04N19/70 take precedence) - H04N 19/70: characterised by syntax aspects related to video coding, e.g. related to compression standards - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A method comprising: encoding a first picture on a first scalability layer and on a lowest temporal sub-layer; encoding a second picture on a second scalability layer and on the lowest temporal sub-layer, wherein the first picture and the second picture represent the same time instant, encoding one or more first syntax elements, associated with the first picture, with a value indicating that a picture type of the first picture is other than a step-wise temporal sub-layer access (STSA) picture; encoding one or more second syntax elements, associated with the second picture, with a value indicating that a picture type of the second picture is a step-wise temporal sub-layer access picture; and encoding at least a third picture on a second scalability layer and on a temporal sub-layer higher than the lowest temporal sub-layer. ## Claims ### Claim 1 A method comprising receiving coded pictures of a first scalability layer; decoding the coded pictures of the first scalability layer; receiving coded pictures of a second scalability layer, the second scalability layer depending on the first scalability layer; selecting a layer access picture on the second scalability layer from the coded pictures of the second scalability layer, wherein the selected layer access picture is a step-wise temporal sub-layer access (STSA) picture on a lowest temporal sub-layer and with a TemporalID equal to 0; ignoring coded pictures on a second scalability layer prior to, in decoding order, the selected layer access picture; decoding the selected layer access picture; and starting step-wise decoding of at least one enhancement layer in response to said at least one enhancement layer containing an intra random access point (IRAP) picture or an STSA picture on the lowest sub-layer, wherein starting step-wise decoding comprises: when the current picture is a temporal sub-layer access (TSA) or STSA picture at a higher sub-layer than the lowest sub-layer and decoding of the next lower sub-layer in the same layer has been started, and decoding of the same sub-layer of all the reference layers of the TSA or STSA picture has been started, the TSA or STSA picture and all pictures at the same sub-layer following the TSA or STSA picture, in decoding order, in the same layer are decoded. ### Claim 2 (depends on 1) The method according claim 1, wherein said selecting comprises concluding that a picture is a step-wise temporal sub-layer access picture on the basis of its Network Access Layer (NAL) unit type indicating the step-wise temporal sub-layer access picture and its temporal sub-layer identifier indicating the lowest temporal sub-layer. ### Claim 3 (depends on 1) The method according to claim 1, the method further comprising starting decoding of the bitstream in response to a base layer containing an IRAP picture or a STSA picture on the lowest sub-layer; and increasing progressively the number of decoded layers and/or the number of decoded temporal sub-layers. ### Claim 4 (depends on 3) The method according to claim 3, wherein starting the step-wise decoding further comprises the following conditional operation: when a current picture is an IRAP picture and decoding of all reference layers of the IRAP picture has been started, the IRAP picture and all pictures following it, in decoding order, in the same layer are decoded. ### Claim 5 A method comprising receiving coded pictures of a first scalability layer; receiving coded pictures of a second scalability layer, the second scalability layer depending on the first scalability layer; selecting a layer access picture on the second scalability layer from the coded pictures of a second scalability layer, wherein the selected layer access picture is a step-wise temporal sub-layer access picture on the lowest temporal sub-layer and with a TemporalID equal to 0, wherein the step-wise temporal sub-layer access picture provides an access point for layer-wise initialization of decoding of a bitstream with one or more temporal sub-layers; ignoring coded pictures on a second scalability layer prior to, in decoding order, the selected layer access picture; and sending the coded pictures of the first scalability layer and the selected layer access picture in a bitstream. ### Claim 6 An apparatus comprising: at least one processor and at least one memory, said at least one memory stored with code thereon, which when executed by said at least one processor, causes an apparatus to perform at least: receiving coded pictures of a first scalability layer; decoding the coded pictures of the first scalability layer; receiving coded pictures of a second scalability layer, the second scalability layer depending on the first scalability layer; selecting a layer access picture on the second scalability layer from the coded pictures of a second scalability layer, wherein the selected layer access picture is a step-wise temporal sub-layer access (STSA) picture on the lowest temporal sub-layer and with a TemporalID equal to 0; ignoring coded pictures on a second scalability layer prior to, in decoding order, the selected layer access picture; decoding the selected layer access picture; and starting step-wise decoding of at least one enhancement layer in response to said at least one enhancement layer containing an intra random access point (IRAP) picture or an STSA picture on the lowest sub-layer, wherein starting step-wise decoding comprises: when the current picture is a temporal sub-layer access (TSA) or STSA picture at a higher sub-layer than the lowest sub-layer and decoding of the next lower sub-layer in the same layer has been started, and decoding of the same sub-layer of all the reference layers of the TSA or STSA picture has been started, the TSA or STSA picture and all pictures at the same sub-layer following the TSA or STSA picture, in decoding order, in the same layer are decoded. ### Claim 7 An apparatus comprising: at least one processor and at least one memory, said at least one memory stored with code thereon, which when executed by said at least one processor, causes an apparatus to perform at least receiving coded pictures of a first scalability layer; receiving coded pictures of a second scalability layer, the second scalability layer depending on the first scalability layer; selecting a layer access picture on the second scalability layer from the coded pictures of a second scalability layer, wherein the selected layer access picture is a step-wise temporal sub-layer access picture on the lowest temporal sub-layer and with a TemporalID equal to 0, wherein the step-wise temporal sub-layer access picture provides an access point for layer-wise initialization of decoding of a bitstream with one or more temporal sub-layers; ignoring coded pictures on a second scalability layer prior to, in decoding order, the selected layer access picture; and sending the coded pictures of the first scalability layer and the selected layer access picture in a bitstream. ### Claim 8 A method comprising encoding a first picture on a first scalability layer and on a lowest temporal sub-layer; encoding a second picture on a second scalability layer and on the lowest temporal sub-layer, wherein the first picture and the second picture represent the same time instant, encoding one or more first syntax elements, associated with the first picture, with a value indicating that a picture type of the first picture is other than a step-wise temporal sub-layer access (STSA) picture; encoding one or more second syntax elements, associated with the second picture, with a value indicating that a picture type of the second picture is a step-wise temporal sub-layer access picture, wherein the step-wise temporal sub-layer access picture is the step-wise temporal sub-layer access picture on a lowest temporal sub-layer and with a TemporalID equal to 0, wherein the step-wise temporal sub-layer access picture provides an access point for layer-wise initialization of decoding of a bitstream with one or more temporal sub-layers; and encoding at least a third picture on a second scalability layer and on a temporal sub-layer higher than the lowest temporal sub-layer. ### Claim 9 (depends on 8) The method according to claim 8, the method further comprising signaling the step-wise temporal sub-layer access picture in the bitstream by a specific Network Access Layer (NAL) unit type. ### Claim 10 (depends on 8) The method according to claim 8, the method further comprising encoding said second or any further scalability layer to comprise more frequent temporal sub-layer access (TSA) or STSA pictures than the first scalability layer. ### Claim 11 An apparatus comprising: at least one processor and at least one memory, said at least one memory stored with code thereon, which when executed by said at least one processor, causes an apparatus to perform at least: encoding a first picture on a first scalability layer and on a lowest temporal sub-layer; encoding a second picture on a second scalability layer and on the lowest temporal sub-layer, wherein the first picture and the second picture represent the same time instant, encoding one or more first syntax elements, associated with the first picture, with a value indicating that a picture type of the first picture is other than a step-wise temporal sub-layer access (STSA) picture; encoding one or more second syntax elements, associated with the second picture, with a value indicating that a picture type of the second picture is a step-wise temporal sub-layer access picture, wherein the step-wise temporal sub-layer access picture is the step-wise temporal sub-layer access picture on a lowest temporal sub-layer and with a TemporalID equal to 0, and wherein the step-wise temporal sub-layer access picture provides an access point for layer-wise initialization of decoding of a bitstream with one or more temporal sub-layers; and encoding at least a third picture on a second scalability layer and on a temporal sub-layer higher than the lowest temporal sub-layer. ### Claim 12 (depends on 11) The apparatus according to claim 11, further comprising code causing the apparatus to perform signaling the step-wise temporal sub-layer access picture in the bitstream by a specific Network Access Layer (NAL) unit type. ### Claim 13 (depends on 11) The apparatus according to claim 11, further comprising code causing the apparatus to perform encoding said second or any further scalability layer to comprise more frequent temporal sub-layer access (TSA) or STSA pictures than the first scalability layer. ### Claim 14 A non-transitory computer readable storage medium stored with code thereon for use by an apparatus, which when executed by a processor, causes the apparatus to perform: encoding a first picture on a first scalability layer and on a lowest temporal sub-layer; encoding a second picture on a second scalability layer and on the lowest temporal sub-layer, wherein the first picture and the second picture represent the same time instant, encoding one or more first syntax elements, associated with the first picture, with a value indicating that a picture type of the first picture is other than a step-wise temporal sub-layer access (STSA) picture; encoding one or more second syntax elements, associated with the second picture, with a value indicating that a picture type of the second picture is a step-wise temporal sub-layer access picture, wherein the step-wise temporal sub-layer access picture is the step-wise temporal sub-layer access picture on a lowest temporal sub-layer and with a TemporalID equal to 0, and wherein the step-wise temporal sub-layer access picture provides an access point for layer-wise initialization of decoding of a bitstream with one or more temporal sub-layers; and encoding at least a third picture on a second scalability layer and on a temporal sub-layer higher than the lowest temporal sub-layer. ### Claim 15 (depends on 14) The non-transitory computer readable storage medium according to claim 14, further comprising code causing the apparatus to perform signaling the step-wise temporal sub-layer access picture in the bitstream by a specific Network Access Layer (NAL) unit type. ### Claim 16 (depends on 14) The non-transitory computer readable storage medium according to claim 14, further comprising code causing the apparatus to perform encoding said second or any further scalability layer to comprise more frequent temporal sub-layer access (TSA) or STSA pictures than the first scalability layer.