US 10,123,027 B2Granted

Method and apparatus for video coding and decoding

AssigneeNokia Technologies Oy

InventorsMiska Matias Hannuksela, Kemal Ugur

Life of this patent

Estimating the term…

Loading patent…

Abstract

Various methods, apparatuses and computer program products for video encoding and decoding are provided. In some embodiments, to indicate a composition of pictures of different time instants, some usability information may be embedded to the video bitstream indicating the intended display behavior when more than one layer is used and associated display behavior using this information. More specifically, the embedded information may indicate how the layers need to be composed in the video bitstream. The information may indicate if the intended display behavior is to combine (overlay) base layer and enhancement layer decoded pictures. The information may indicate that the base and enhancement layer decoded pictures are allowed to be overlaid. In addition or alternatively, the information may indicate that the enhancement layer picture should not be combined with base layer picture during display.

Claims14

Dependent claims

A method comprising:

determining an intended displaying behavior of at least two scalability layers of a bitstream, the at least two scalability layers comprising pictures;
encoding, into the bitstream, a description of the bitstream, and an indication, which indicates the intended displaying behavior, when the at least two scalability layers are output layers for decoding the bitstream;
and
performing at least one of the following:
(i) encoding pictures into the bitstream, the bitstream comprising the at least two scalability layers, (ii) or encapsulating the bitstream into a container file, wherein the encoded pictures at least partially represent a same scene and the indication of the intended displaying behavior indicates an intent for one or more of overlaying pictures of the at least two scalability layers, or displaying the pictures of the at least two scalability layers separately, wherein pictures are associated with access units, and in the absence of a picture at the output layer within an access unit, the indication indicates that the intended displaying behavior is:
combining pictures at output layers that are present in an access unit and at least one of the following:
another picture of the output layer that is not present in the access unit;
an interpolated picture that is obtained by interpolating other pictures in the output layer, or
a synthesized picture that is obtained by depth-image-based rendering, and
wherein in an instance in which the indication indicates that the intended displaying behavior is overlaying pictures of the at least two scalability layers, the method further comprises at least one of the following:
(i) encoding a picture of the first output layer;
or (ii) encoding a picture of the second output layer.

9 more elements

A method comprising:

decoding pictures from a bitstream, the bitstream comprising at least two scalability layers and pictures associated with access units;
decoding from the bitstream, a container file encapsulating the bitstream, a description of the bitstream, and an indication, which indicates an intended displaying behavior, when the at least two scalability layers are output layers;
and
displaying the at least two scalability layers according to the indication,
wherein the decoded pictures at least partially represent a same scene and the indication of the intended displaying behavior indicates an intent for one or more of overlaying pictures of the at least two scalability layers, or displaying the pictures of the at least two scalability layers separately, wherein, in the absence of a picture at the output layer within an access unit, and in response to the indication, the method comprises:
combining pictures at output layers that are present in an access unit and at least one of the following:
another picture of the output layer that is not present in the access unit;
an interpolated picture that is obtained by interpolating other pictures in the output layer;
or
a synthesized picture that is obtained by depth-image-based rendering, and
wherein the method further comprises determining on the basis of the indication that the intended displaying behavior is overlaying pictures of the at least two scalability layers, and at least one of the following:
(i) decoding a picture of the first output layer and displaying the decoded picture of the first output layer;
or (ii) decoding a picture of the second output layer and displaying the decoded picture of the second output layer.

10 more elements

An apparatus comprising at least one processor and at least one memory, said at least one memory stored with code thereon, which when executed by said at least one processor, causes an apparatus to perform at least the following:

determine an intended displaying behavior of at least two scalability layers of a bitstream, the at least two scalability layers comprising pictures;
encode, into the bitstream, a description of the bitstream, and an indication, which indicates the intended displaying behavior, when the at least two scalability layers are output layers for decoding the bitstream;
and
perform at least one of the following:
(i) encode pictures into the bitstream, the bitstream comprising at least two scalability layers or (ii) encapsulate the bitstream into a container file, wherein the encoded pictures at least partially represent a same scene and the indication indicates an intent for one or more of overlaying pictures of the at least two scalability layers, or displaying the pictures of the at least two scalability layers separately, wherein pictures are associated with access units, and in the absence of a picture at the output layer within an access unit, the indication indicates that the intended displaying behavior is:
combining pictures at output layers that are present in an access unit and at least one of the following:
another picture of the output layer that is not present in the access unit;
an interpolated picture that is obtained by interpolating other pictures in the output layer, or
a synthesized picture that is obtained by depth-image-based rendering,
wherein in an instance in which the indication indicates that the intended displaying behavior is overlaying pictures of the at least two scalability layers, the at least one memory stored with code thereon, when executed by said at least one processor, further causes the apparatus to perform at least one of the following:
(i) encode a picture of the first output layer;
or (ii) encode a picture of the second output layer.

9 more elements

decode pictures from a bitstream, the bitstream comprising at least two scalability layers and pictures associated with access units, wherein one of the at least two layers is an output layer and another of the at least two layers is an alternative layer;
decode from the bitstream, a container file encapsulating the bitstream, a description of the bitstream, and an indication, which indicates an intended displaying behavior, when the at least two scalability layers are output layers of an output layer set;
and
output a picture of the output layer, a picture of the alternative layer, or a combined picture, wherein the decoded pictures at least partially represent a same scene and the indication of the intended displaying behavior indicates an intent for one or more of overlaying pictures of the at least two scalability layers, or displaying the pictures of the at least two scalability layers separately, wherein, in the absence of a picture at the output layer within an access unit, and in response to the indication, the apparatus is further caused to perform the following:
combining pictures at output layers that are present in an access unit and at least one of the following:
another picture of the output layer that is not present in the access unit;
an interpolated picture that is obtained by interpolating other pictures in the output layer;
or a synthesized picture that is obtained by depth-image-based rendering,
wherein said at least one memory stored with code thereon, when executed by said at least one processor, further causes the apparatus to determine on the basis of the indication that the intended displaying behavior is overlaying pictures of the at least two scalability layers, and at least one of the following:
(i) decode a picture of the first output layer and display the decoded picture of the first output layer;
or (ii) decode a picture of the second output layer and display the decoded picture of the second output layer.

8 more elements

A computer program product embodied on a non-transitory computer readable medium, comprising computer program code configured to, when executed on at least one processor, cause an apparatus or a system to:

determine an intended displaying behavior of at least two scalability layers of a bitstream, the at least two scalability layers comprising pictures;
encode, into the bitstream, a description of the bitstream, and an indication, which indicates the intended displaying behavior, when the at least two scalability layers are output layers in an output layer set for decoding the bitstream;
and
perform at least one of the following:
(i) encode pictures into the bitstream, the bitstream comprising at least two scalability layers or (ii) encapsulate the bitstream into a container file, wherein the encoded pictures at least partially represent a same scene and the indication of the intended displaying behavior indicates an intent for one or more of overlaying pictures of the at least two scalability layers, or displaying the pictures of the at least two scalability layers separately, wherein pictures are associated with access units, and in the absence of a picture at the output layer within an access unit, the indication indicates that the intended displaying behavior is:
combining pictures at output layers that are present in an access unit and at least one of the following:
another picture of the output layer that is not present in the access unit;
an interpolated picture that is obtained by interpolating other pictures in the output layer, or
a synthesized picture that is obtained by depth-image-based rendering,
wherein in an instance in which the indication indicates that the intended displaying behavior is overlaying pictures of the at least two scalability layers, the computer program code is configured to, when executed on the at least one processor, further cause the apparatus or the system to perform at least one of the following:
(i) encode a picture of the first output layer;
or (ii) encode a picture of the second output layer.

9 more elements

A computer program product stored on a non-transitory computer readable medium comprising at least one processor and at least one memory, said at least one memory stored with code thereon, which when executed by said at least one processor, causes an apparatus or a system to perform at least the following:

decode pictures from a bitstream, the bitstream comprising at least two scalability layers and pictures associated with access units, wherein one of the at least two layers is an output layer and another of the at least two layers is an alternative layer;
decode from the bitstream, a container file encapsulating the bitstream, a description of the bitstream, and an indication, which indicates an intended displaying behavior, when the at least two scalability layers are output layers in an output layer set;
and
output a picture of the output layer, a picture of the alternative layer, or a combined picture, wherein the decoded pictures at least partially represent a same scene and the indication of the intended displaying behavior indicates an intent for one or more of overlaying pictures of the at least two scalability layers, or displaying the pictures of the at least two scalability layers separately, wherein, in the absence of a picture at the output layer within an access unit, and in response to the indication, the apparatus or the system is further caused to perform the following:
combining pictures at output layers that are present in an access unit and at least one of the following:
another picture of the output layer that is not present in the access unit;
an interpolated picture that is obtained by interpolating other pictures in the output layer;
or a synthesized picture that is obtained by depth-image-based rendering,
wherein the computer program code is configured to, when executed on the at least one processor, further cause the apparatus or the system to determine on the basis of the indication that the intended displaying behavior is overlaying pictures of the at least two scalability layers, and at least one of the following:
(i) decode a picture of the first output layer and display the decoded picture of the first output layer;
or (ii) decode a picture of the second output layer and display the decoded picture of the second output layer.

8 more elements

Dependencies are inferred from the claim text.

Facts

Application: 14/583,371Filed 26 Dec 2014 · us provisional application
Priority: 7 Jan 2014
Published: US 2015/0304665 A122 Oct 2015
Granted: 6 Nov 2018
Term ends: Not estimatedThe source records don’t support an estimate.

Classification6 codes

H04N 19/187
Characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the…: the…
H04N 13/161
Processing image signals: encoding, multiplexing or demultiplexing different image signal components
H04N 19/30
Methods or arrangements for coding, decoding, compressing or decompressing digital video signals: using hierarchical techniques, e.g. scalability
H04N 19/463
Embedding additional information in the video signal during the compression process: by compressing encoding parameters before transmission

Family

Application61/924,516Us provisional application · filed 7 Jan 2014
US 2015/0304665 A1Pre-grant publication · published 22 Oct 2015
US 10,123,027 B2This patent

From the US filing history. Foreign family members aren’t listed.

Timeline

Citations

Open elsewhere

Building an agent or pulling many patents? Use the API or MCP server instead of scraping pages. Get an API key