# US 2020/0288151 A1 · Arrangements and methods thereof for processing video > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2020/0288151 A1 - Document ID: US20200288151A1 - Status: Published application - Kind: A1 - Application number: 16/883,252 - Filed: 2020-05-26 - Published: 2020-09-10 - Earliest claimed: 2012-04-16 - Assignees: Telefonaktiebolaget L M Ericsson (publ) - Applicants: Telefonaktiebolaget L M Ericsson (publ) - Inventors: Rickard SJÖBERG; Jonatan SAMUELSSON - Web page: https://patentgrep.com/patent/US20200288151A1 ## CPC - H04N 19/177: the unit being a group of pictures [GOP] - H04N 19/31: in the temporal domain - H04N 19/573: Motion compensation with multiple frame prediction using two or more reference frames in a given prediction direction - H04N 19/503: involving temporal prediction (adaptive coding with adaptive selection between spatial and temporal predictive coding H04N19/107; adaptive coding with adaptive selection among a plurality of temporal predictive coding modes H04N19/109) - H04N 19/70: characterised by syntax aspects related to video coding, e.g. related to compression standards - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A method performed by a video encoder for encoding a current picture belonging to a temporal level identified by a temporal_id. The method includes determining a Reference Picture Set (RPS) for the current picture indicating reference pictures that are kept in a decoded picture buffer (DPB) when decoding the current picture, and when the current picture is a temporal switching point. The method further comprises operating to ensure that the RPS of the current picture includes no picture having a temporal_id greater than or equal to the temporal_id of the current picture. ## Claims ### Claim 1 A video encoder for encoding a current picture belonging to a temporal level identified by a temporal_id, the video encoder comprising: at least one processor; and at least one memory storing program code that is executed by the at least one processor to perform operations comprising: determining a Reference Picture Set (RPS) for the current picture indicating reference pictures that are used when decoding the current picture; and encoding the current picture ensuring that the RPS of the current picture includes no picture having a temporal_id greater than or equal to the temporal_id of the current picture when the current picture is a temporal switching point, wherein the encoding is configured when the current picture is a temporal switching point and is a temporal sub-layer access (TSA) picture to trigger a parsing circuit performing parsing of bits representing at least a part of the current picture to determine when coded pictures with temporal_id greater than or equal to the temporal_id of the TSA picture that follow the TSA picture in decoding order include any picture in their reference picture set that precedes the TSA picture in decoding order and for which temporal_id is greater than or equal to the temporal_id of the TSA picture. ### Claim 2 (depends on 1) The video encoder according to claim 1, wherein the encoding is operative to identify a bit-error based on determining that coded pictures with temporal_id greater than or equal to the temporal_id of the TSA picture that follow the TSA picture in decoding order include a picture in their reference picture set that precedes the TSA picture in decoding order and for which temporal_id is greater than or equal to the temporal_id of the TSA picture. ### Claim 3 (depends on 1) The video encoder according to claim 1, wherein the encoding is operative to identify a loss of data based on determining that coded pictures with temporal_id greater than or equal to the temporal_id of the TSA picture that follow the TSA picture in decoding order include a picture in their reference picture set that precedes the TSA picture in decoding order and for which temporal_id is greater than or equal to the temporal_id of the TSA picture. ### Claim 4 (depends on 1) The video encoder according to claim 1, wherein the encoding is operative to identify a non-compliant bitstream based on determining that coded pictures with temporal_id greater than or equal to the temporal_id of the TSA picture that follow the TSA picture in decoding order include a picture in their reference picture set that precedes the TSA picture in decoding order and for which temporal_id is greater than or equal to the temporal_id of the TSA picture. ### Claim 5 (depends on 1) The video encoder according to claim 1, wherein the temporal switching point is a coded picture for which each slice has a unique nal_unit_type. ### Claim 6 (depends on 5) The video encoder according to claim 5, wherein the TSA picture is a coded picture for which each slice has a unique nal_unit_type equal to 3. ### Claim 7 (depends on 1) The video encoder according to claim 1, wherein the video encoder is a High Efficiency Video Coding (HEVC) encoder. ### Claim 8 A parsing circuit for parsing bits representing at least a part of a current picture, wherein the current picture belongs to a temporal level identified by a temporal_id, wherein the current picture is associated with a Reference Picture Set (RPS) indicating reference pictures that are used when decoding the current picture, the parsing circuit comprising: at least one processor; and at least one memory storing program code that is executed by the at least one processor to perform operations comprising: determining that for the current picture, when being a temporal switching point, whether there is a picture included in the RPS of the current picture that has the same or higher temporal_id as the current picture; and when the current picture is a temporal switching point and is a temporal sub-layer access (TSA) picture, determine when coded pictures with temporal_id greater than or equal to the temporal_id of the TSA picture that follow the TSA picture in decoding order include any picture in their reference picture set that precedes the TSA picture in decoding order and for which temporal_id is greater than or equal to the temporal_id of the TSA picture. ### Claim 9 (depends on 8) The parsing circuit according to claim 8, wherein the operations further comprise: controlling decoding operations of the current picture responsive to determining that coded pictures with temporal_id greater than or equal to the temporal_id of the TSA picture that follow the TSA picture in decoding order include a picture in their reference picture set that precedes the TSA picture in decoding order and for which temporal_id is greater than or equal to the temporal_id of the TSA picture. ### Claim 10 (depends on 9) The parsing circuit according to claim 9, wherein the operations further comprise: identifying that a bit-error, loss of data, or non-compliant bitstream has occurred responsive to determining that coded pictures with temporal_id greater than or equal to the temporal_id of the TSA picture that follow the TSA picture in decoding order include a picture in their reference picture set that precedes the TSA picture in decoding order and for which temporal_id is greater than or equal to the temporal_id of the TSA picture. ### Claim 11 (depends on 8) The parsing circuit according to claim 8, wherein the temporal switching point is a coded picture for which each slice has a unique nal_unit_type. ### Claim 12 (depends on 11) The parsing circuit according to claim 11, wherein the TSA picture is a coded picture for which each slice has a unique nal_unit_type equal to 3. ### Claim 13 (depends on 8) The parsing circuit according to claim 8, wherein the parsing circuit is a network element. ### Claim 14 (depends on 8) The parsing circuit according to claim 8, wherein the parsing circuit is a decoder, wherein the decoder is a High Efficiency Video Coding (HEVC) decoder. ### Claim 15 A computer program product for encoding a current picture belonging to a temporal level identified by a temporal_id, the computer program product comprising: at least one non-transitory computer readable medium storing program code that is executable by at least one processor to perform operations comprising: determining a Reference Picture Set (RPS) for the current picture indicating reference pictures that are used when decoding the current picture; and encoding the current picture ensuring that the RPS of the current picture includes no picture having a temporal_id greater than or equal to the temporal_id of the current picture when the current picture is a temporal switching point, wherein the encoding is configured when the current picture is a temporal switching point and is a temporal sub-layer access (TSA) picture to trigger a parsing circuit performing parsing of bits representing at least a part of the current picture to determine when coded pictures with temporal_id greater than or equal to the temporal_id of the TSA picture that follow the TSA picture in decoding order include any picture in their reference picture set that precedes the TSA picture in decoding order and for which temporal_id is greater than or equal to the temporal_id of the TSA picture. ### Claim 16 (depends on 15) The computer program product according to claim 15, wherein the encoding is operative to identify a bit-error based on determining that coded pictures with temporal_id greater than or equal to the temporal_id of the TSA picture that follow the TSA picture in decoding order include a picture in their reference picture set that precedes the TSA picture in decoding order and for which temporal_id is greater than or equal to the temporal_id of the TSA picture. ### Claim 17 (depends on 15) The computer program product according to claim 15, wherein the encoding is operative to identify a loss of data based on determining that coded pictures with temporal_id greater than or equal to the temporal_id of the TSA picture that follow the TSA picture in decoding order include a picture in their reference picture set that precedes the TSA picture in decoding order and for which temporal_id is greater than or equal to the temporal_id of the TSA picture. ### Claim 18 (depends on 15) The computer program product according to claim 15, wherein the encoding is operative to identify a non-compliant bitstream based on determining that coded pictures with temporal_id greater than or equal to the temporal_id of the TSA picture that follow the TSA picture in decoding order include a picture in their reference picture set that precedes the TSA picture in decoding order and for which temporal_id is greater than or equal to the temporal_id of the TSA picture. ### Claim 19 (depends on 15) The computer program product according to claim 15, wherein the temporal layer switching point is a coded picture for which each slice has a unique nal_unit_type. ### Claim 20 (depends on 15) The computer program product according to claim 15, wherein the TSA picture is a coded picture for which each slice has a unique nal_unit_type equal to 3. ### Claim 21 A computer program product for parsing bits representing at least a part of a current picture, wherein the current picture belongs to a temporal level identified by a temporal_id, wherein the current picture is associated with a Reference Picture Set (RPS) indicating reference pictures to be used when decoding the current picture, the computer program product comprising: at least one non-transitory computer readable medium storing program code that is executable by at least one processor to perform operations comprising: determining that for the current picture, when being a temporal switching point, whether there is a picture included in the RPS of the current picture that has the same or higher temporal_id as the current picture; and when the current picture is a temporal switching point and is a temporal sub-layer access (TSA) picture, determine when coded pictures with temporal_id greater than or equal to the temporal_id of the TSA picture that follow the TSA picture in decoding order include any picture in their reference picture set that precedes the TSA picture in decoding order and for which temporal_id is greater than or equal to the temporal_id of the TSA picture. ### Claim 22 (depends on 21) The computer program product according to claim 21, wherein the operations further comprise: controlling decoding operations of the current picture responsive to determining that coded pictures with temporal_id greater than or equal to the temporal_id of the TSA picture that follow the TSA picture in decoding order include a picture in their reference picture set that precedes the TSA picture in decoding order and for which temporal_id is greater than or equal to the temporal_id of the TSA picture. ### Claim 23 (depends on 22) The computer program product according to claim 22, wherein the operations further comprise: identifying that a bit-error, loss of data, or non-compliant bitstream has occurred responsive to determining that coded pictures with temporal_id greater than or equal to the temporal_id of the TSA picture that follow the TSA picture in decoding order include a picture in their reference picture set that precedes the TSA picture in decoding order and for which temporal_id is greater than or equal to the temporal_id of the TSA picture. ### Claim 24 (depends on 21) The computer program product according to claim 21, wherein the temporal switching point is a coded picture for which each slice has a unique nal_unit_type. ### Claim 25 (depends on 24) The computer program product according to claim 24, wherein the TSA picture is a coded picture for which each slice has a unique nal_unit_type equal to 3. ### Claim 26 (depends on 21) The computer program product according to claim 21, wherein the at least one processor that executes the program code is within a network element.