# US 11,909,983 B2 · Apparatus, a method and a computer program for video coding and decoding > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,909,983 B2 - Document ID: US11909983B2 - Status: Granted - Kind: B2 - Application number: 17/433,034 - Filed: 2020-02-19 - Published: 2022-05-19 - Granted: 2024-02-20 - Earliest claimed: 2019-03-14 - Assignees: Nokia Technologies Oy - Applicants: Nokia Technologies Oy - Inventors: Miska Hannuksela; Igor Curcio; Sujeet Shyamsundar Mate; Emre Aksu; Ari Hourunranta - Web page: https://patentgrep.com/patent/US11909983B2 ## CPC - H04N 19/146: Data rate or code amount at the encoder output - H04N 21/816: {involving special video data, e.g 3D video} - H04N 19/105: Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction - H04N 19/107: between spatial and temporal predictive coding, e.g. picture refresh - H04N 19/154: Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion (use of rate-distortion criteria H04N19/147) - H04N 19/159: Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction - H04N 19/172: the region being a picture, frame or field - H04N 19/177: the unit being a group of pictures [GOP] - H04N 19/39: involving multiple description coding [MDC], i.e. with separate layers being structured as independently decodable descriptions of input picture data - H04N 19/46: Embedding additional information in the video signal during the compression process (H04N19/517, H04N19/68, H04N19/70 take precedence) - H04N 19/597: specially adapted for multi-view video sequence encoding - H04N 19/70: characterised by syntax aspects related to video coding, e.g. related to compression standards - H04N 21/234381: {by altering the temporal resolution, e.g. decreasing the frame rate by frame skipping} - H04N 21/23439: {for generating different versions} - H04N 21/2393: {involving handling client requests (scheduling or organising the servicing of application requests H04L67/60)} - H04N 21/4728: for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region - H04N 21/6587: Control parameters, e.g. trick play commands, viewpoint selection - H04N 19/31: in the temporal domain - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A method comprising: encoding video media content into at least a first representation and a second representation, wherein the first representation comprises a switchable inter-coded picture, wherein the switchable picture has a first characteristic that no pictures preceding the switchable picture in decoding order are used as reference pictures for any non-switchable pictures following the switchable picture in decoding order; and the second representation comprises one or more decoder reset pictures aligned with the switchable pictures of the first representation, wherein the frame rate of the second representation is smaller than or equal to the frame rate of the first representation and the resolutions of the first and the second representation are equal. ## Claims ### Claim 1 A method comprising: encoding video media content into at least a first representation and a second representation, wherein: the first representation comprises a switchable inter-coded picture, wherein the switchable inter-coded picture comprises a first characteristic that no pictures preceding the switchable inter-coded picture in decoding order are used as reference pictures for non-switchable pictures following the switchable inter-coded picture in decoding order; and the second representation comprises one or more decoder reset pictures aligned with switchable inter-coded pictures of the first representation, wherein a frame rate of the second representation is smaller than or equal to a frame rate of the first representation and resolutions of the first and the second representations are equal or substantially equal, wherein the first representation and the second representation are encoded to enable decoding of a decoder reset picture of the second representation, followed by decoding of pictures of the first representation that are subsequent to a switchable inter-coded picture of the first representation that corresponds to the decoder reset picture, without the corresponding switchable inter-coded picture having been decoded. ### Claim 2 An apparatus comprising at least one processor and at least one non-transitory memory including computer program code for one or more programs, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus at least to: encode video media content into at least a first representation and a second representation, wherein: the first representation comprises a switchable inter-coded picture, wherein the switchable inter-coded picture comprises a first characteristic that no pictures preceding the switchable inter-coded picture in decoding order are used as reference pictures for non-switchable pictures following the switchable inter-coded picture in decoding order; and the second representation comprises one or more decoder reset pictures aligned with switchable inter-coded pictures of the first representation, wherein a frame rate of the second representation is smaller than or equal to a frame rate of the first representation and resolutions of the first and the second representations are equal or substantially equal, wherein the first representation and the second representation are encoded to enable decoding of a decoder reset picture of the second representation, followed by decoding of pictures of the first representation that are subsequent to a switchable inter-coded picture of the first representation that corresponds to the decoder reset picture, without the corresponding switchable inter-coded picture having been decoded. ### Claim 3 (depends on 2) The apparatus according to claim 2, wherein the switchable inter-coded picture comprises a second characteristic that the switchable inter-coded picture is predicted at least from zero or more previous switchable inter-coded pictures or from a previous random access picture in decoding order. ### Claim 4 (depends on 2) The apparatus according to claim 2, wherein the apparatus is further caused to indicate, in or along a bitstream corresponding to the first representation, suitability of content of the first and second representations for switching between the first representation and the second representation in decoding. ### Claim 5 (depends on 2) The apparatus according to claim 2, wherein the decoder reset picture is configured to be encoded as a broken link access picture type comprising no leading pictures following the broken link access picture or as an independent decoding refresh picture. ### Claim 6 A method comprising: receiving at least one bitstream corresponding to a first representation of encoded video media content, the first representation comprising a switchable inter-coded picture, wherein the switchable inter-coded picture comprises a first characteristic that no pictures preceding the switchable inter-coded picture in decoding order are used as reference pictures for non-switchable pictures following the switchable inter-coded picture in decoding order; receiving, from or along the at least one bitstream, an indication about suitability of switching between the first representation and a second representation of the encoded video media content in decoding, wherein the second representation comprises one or more decoder reset pictures aligned with switchable inter-coded pictures of the first representation, wherein a frame rate of the second representation is smaller than or equal to a frame rate of the first representation and resolutions of the first and the second representations are equal or substantially equal; receiving at least one decoder reset picture of the second representation; and decoding the at least one decoder reset picture of the second representation, followed by decoding of pictures of the first representation that are subsequent to a switchable inter-coded picture of the first representation that corresponds to the at least one decoder reset picture, without the corresponding switchable inter-coded picture having been decoded. ### Claim 7 An apparatus comprising at least one processor and at least one non-transitory memory including computer program code for one or more programs, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus at least to: receive at least one bitstream corresponding to a first representation of encoded video media content, the first representation comprising a switchable inter-coded picture, wherein the switchable inter-coded picture comprises a first characteristic that no pictures preceding the switchable inter-coded picture in decoding order are used as reference pictures for non-switchable pictures following the switchable inter-coded picture in decoding order; receive, from or along the at least one bitstream, an indication about suitability of switching between the first representation and a second representation of the encoded video media content in decoding, wherein the second representation comprises one or more decoder reset pictures aligned with switchable inter-coded pictures of the first representation, wherein a frame rate of the second representation is smaller than or equal to a frame rate of the first representation and resolutions of the first and the second representations are equal or substantially equal; receive at least one decoder reset picture of the second representation; and decode at least one decoder reset picture of the second representation, followed by decoding of pictures of the first representation that are subsequent to a switchable inter-coded picture of the first representation that corresponds to the at least one decoder reset picture, without the corresponding switchable inter-coded picture having been decoded. ### Claim 8 (depends on 7) The apparatus according to claim 7, wherein the switchable inter-coded picture comprises a second characteristic that the switchable inter-coded picture is predicted at least from zero or more previous switchable inter-coded pictures or from a previous random access picture in decoding order. ### Claim 9 (depends on 7) The apparatus according to claim 7, wherein the apparatus is further caused to request, in response to a pause in a playback of the decoded first representation, the at least one decoder reset picture of the second representation. ### Claim 10 (depends on 7) The apparatus according to claim 7, wherein the apparatus is further caused to request, in response to a viewport switch, the at least one decoder reset picture of the second representation. ### Claim 11 (depends on 10) The apparatus according to claim 10, wherein the viewport switch takes place from a first viewport to a second viewport, and wherein the first representation covers the first viewport, and wherein the apparatus is further caused to receive a second bitstream corresponding to a third representation of the encoded video media content, the third representation representing the second viewport. ### Claim 12 (depends on 7) The apparatus according to claim 7, wherein the apparatus is further caused to request, in response to a viewpoint switch, the at least one decoder reset picture of the second representation. ### Claim 13 (depends on 12) The apparatus according to claim 12, wherein the viewpoint switch takes place from a first viewpoint to a second viewpoint, and wherein the first representation covers the first viewpoint, and wherein the apparatus is further caused to receive a second bitstream corresponding to a third representation of the encoded video media content, the third representation representing the second viewpoint. ### Claim 14 (depends on 7) The apparatus according to claim 7, wherein the apparatus is further caused to: decode, in response to a re-start of a playback, a decoder reset picture substantially timely corresponding to the re-start; and switch to decode pictures of the first representation following, in decoding order, a switchable inter-coded picture of the first representation, which is temporarily aligned with the decoder reset picture substantially timely corresponding to the re-start. ### Claim 15 (depends on 7) The apparatus according to claim 7, wherein the apparatus is further caused to: request, in response to a pause in a playback of the decoded first representation, a plurality of decoder reset pictures of the second representation based on available bandwidth. ### Claim 16 (depends on 7) The apparatus according to claim 7, wherein the apparatus is further caused to: request the at least one decoder reset picture of the second representation for carrying out bitrate adaptation between the first representation and a third representation. ### Claim 17 (depends on 16) The apparatus according to claim 16, wherein a bitrate switch takes place from a first bitrate to a second bitrate, and wherein the first representation corresponds to the first bitrate, and wherein the apparatus is further caused to receive a second bitstream corresponding to the third representation of the encoded video media content, the third representation representing the second bitrate. ### Claim 18 (depends on 7) The apparatus according to claim 7, wherein the apparatus is further caused to: extract at least one switchable inter-coded picture from a first extractor track or representation; and extract the at least one decoder reset picture from a second extractor track or representation. ### Claim 19 A computer program product comprises at least one non-transitory computer-readable storage medium having computer executable program code instructions stored therein, the computer executable program code instructions comprising program code instructions configured, upon execution, to: receive at least one bitstream corresponding to a first representation of encoded video media content, the first representation comprising a switchable inter-coded picture, wherein the switchable inter-coded picture comprises a first characteristic that no pictures preceding the switchable inter-coded picture in decoding order are used as reference pictures for non-switchable pictures following the switchable inter-coded picture in decoding order; receive, from or along the at least one bitstream, an indication about suitability of switching between the first representation and a second representation of the encoded video media content in decoding, wherein the second representation comprises one or more decoder reset pictures aligned with switchable inter-coded pictures of the first representation, wherein the frame rate of the second representation is smaller than or equal to a frame rate of the first representation and resolutions of the first and the second representation are equal or substantially equal; receive at least one decoder reset picture of the second representation; and decode the at least one decoder reset picture of the second representation, followed by decoding of pictures of the first representation that are subsequent to a switchable inter-coded picture of the first representation that corresponds to the at least one decoder reset picture, without the corresponding switchable inter-coded picture having been decoded.