Back to results
US 10,484,682 B2Granted
Reference picture derivation and motion compensation for 360-degree video coding
AssigneeQUALCOMM Incorporated
InventorsMuhammed Zeyd Coban, Geert Van der Auwera, Marta Karczewicz
Life of this patent
Estimating the term…
Abstract
This disclosure describes techniques for generating reference frames packed with extended faces from a cubemap projection or adjusted cubemap projection of 360-degree video data. The reference frames packed with the extended faces may be used for inter-prediction of subsequent frames of 360-degree video data.
Claims30
A method of decoding 360-degree video data, the method comprising:
- receiving an encoded frame of 360-degree video data, the encoded frame of 360-degree video data being arranged in packed faces obtained from a projection of a sphere of the 360-degree video data;
- decoding the frame of encoded 360-degree video data to obtain a decoded frame of 360-degree video data, the decoded frame of 360-degree video data being arranged in the packed faces;
- deriving a decoded sphere of 360-degree video data from the decoded frame of 360-degree video data;
- sampling the decoded sphere of 360-degree video data to produce extended faces using the projection, wherein the extended faces are larger than the packed faces of the decoded frame of 360-degree video data;
- deriving an extended reference frame from the extended faces;
- and
- decoding a subsequent encoded frame of 360-degree video data using an inter-prediction process and the derived extended reference frame.
4 more elements
An apparatus configured to decode 360-degree video data, the apparatus comprising:
- a memory configured to store an encoded frame of 360-degree video data;
- and
- one or more processors in communication with the memory, the one or more processors configured to:
- receive the encoded frame of 360-degree video data, the encoded frame of 360-degree video data being arranged in packed faces obtained from a projection of a sphere of the 360-degree video data;
- decode the frame of encoded 360-degree video data to obtain a decoded frame of 360-degree video data, the decoded frame of 360-degree video data being arranged in the packed faces;
- derive a decoded sphere of 360-degree video data from the decoded frame of 360-degree video data;
- sample the decoded sphere of 360-degree video data to produce extended faces using the projection, wherein the extended faces are larger than the packed faces of the decoded frame of 360-degree video data;
- derive an extended reference frame from the extended faces;
- and
- decode a subsequent encoded frame of 360-degree video data using an inter-prediction process and the derived extended reference frame.
7 more elements
An apparatus configured to decode 360-degree video data, the apparatus comprising:
- means for receiving an encoded frame of 360-degree video data, the encoded frame of 360-degree video data being arranged in packed faces obtained from a projection of a sphere of the 360-degree video data;
- means for decoding the frame of encoded 360-degree video data to obtain a decoded frame of 360-degree video data, the decoded frame of 360-degree video data being arranged in the packed faces;
- means for deriving a decoded sphere of 360-degree video data from the decoded frame of 360-degree video data;
- means for sampling the decoded sphere of 360-degree video data to produce extended faces using the projection, wherein the extended faces are larger than the packed faces of the decoded frame of 360-degree video data;
- means for deriving an extended reference frame from the extended faces;
- and
- means for decoding a subsequent encoded frame of 360-degree video data using an inter-prediction process and the derived extended reference frame.
4 more elements
A computer-readable storage medium storing instructions that, when executed, causes one or more processors of a device configured to decode video data to:
- receive the encoded frame of 360-degree video data, the encoded frame of 360-degree video data being arranged in packed faces obtained from a projection of a sphere of the 360-degree video data;
- decode the frame of encoded 360-degree video data to obtain a decoded frame of 360-degree video data, the decoded frame of 360-degree video data being arranged in the packed faces;
- derive a decoded sphere of 360-degree video data from the decoded frame of 360-degree video data;
- sample the decoded sphere of 360-degree video data to produce extended faces using the projection, wherein the extended faces are larger than the packed faces of the decoded frame of 360-degree video data;
- derive an extended reference frame from the extended faces;
- and
- decode a subsequent encoded frame of 360-degree video data using an inter-prediction process and the derived extended reference frame.
4 more elements
A method of encoding 360-degree video data, the method comprising:
- receiving a sphere of 360-degree video data;
- arranging the sphere of 360-degree video data into a frame of packed faces obtained from a projection of the sphere of 360-degree video data;
- encoding the frame of packed faces to form a frame of encoded 360-degree video data;
- reconstructing the frame of encoded 360-degree video data to obtain a reconstructed frame of 360-degree video data, the reconstructed frame of 360-degree video data being arranged in the packed faces;
- deriving a reconstructed sphere of 360-degree video data from the reconstructed frame of 360-degree video data;
- sampling the reconstructed sphere of 360-degree video data to produce extended faces using the projection, wherein the extended faces are larger than the packed faces of the reconstructed frame of 360-degree video data;
- deriving an extended reference frame from the extended faces;
- and
- encoding a subsequent frame of 360-degree video data using an inter-prediction process and the derived extended reference frame.
6 more elements
An apparatus configured to encode 360-degree video data, the apparatus comprising:
- a memory configured to store a sphere of 360-degree video data;
- and
- one or more processors in communication with the memory, the one or more processors configured to:
- receive the sphere of 360-degree video data;
- arrange the sphere of 360-degree video data into a frame of packed faces obtained from a projection of the sphere of 360-degree video data;
- encode the frame of packed faces to form a frame of encoded 360-degree video data;
- reconstruct the frame of encoded 360-degree video data to obtain a reconstructed frame of 360-degree video data, the reconstructed frame of 360-degree video data being arranged in the packed faces;
- derive a reconstructed sphere of 360-degree video data from the reconstructed frame of 360-degree video data;
- sample the reconstructed sphere of 360-degree video data to produce extended faces using the projection, wherein the extended faces are larger than the packed faces of the reconstructed frame of 360-degree video data;
- derive an extended reference frame from the extended faces;
- and
- encode a subsequent frame of 360-degree video data using an inter-prediction process and the derived extended reference frame.
9 more elements
An apparatus configured to encode 360-degree video data, the apparatus comprising:
- means for receiving a sphere of 360-degree video data;
- means for arranging the sphere of 360-degree video data into a frame of packed faces obtained from a projection of the sphere of 360-degree video data;
- means for encoding the frame of packed faces to form a frame of encoded 360-degree video data;
- means for reconstructing the frame of encoded 360-degree video data to obtain a reconstructed frame of 360-degree video data, the reconstructed frame of 360-degree video data being arranged in the packed faces;
- means for deriving a reconstructed sphere of 360-degree video data from the reconstructed frame of 360-degree video data;
- means for sampling the reconstructed sphere of 360-degree video data to produce extended faces using the projection, wherein the extended faces are larger than the packed faces of the reconstructed frame of 360-degree video data;
- means for deriving an extended reference frame from the extended faces;
- and
- means for encoding a subsequent frame of 360-degree video data using an inter-prediction process and the derived extended reference frame.
6 more elements
A computer-readable storage medium storing instructions that, when executed, causes one or more processors of a device configured to encode video data to:
- receive a sphere of 360-degree video data;
- arrange the sphere of 360-degree video data into a frame of packed faces obtained from a projection of the sphere of 360-degree video data;
- encode the frame of packed faces to form a frame of encoded 360-degree video data;
- reconstruct the frame of encoded 360-degree video data to obtain a reconstructed frame of 360-degree video data, the reconstructed frame of 360-degree video data being arranged in the packed faces;
- derive a reconstructed sphere of 360-degree video data from the reconstructed frame of 360-degree video data;
- sample the reconstructed sphere of 360-degree video data to produce extended faces using the projection, wherein the extended faces are larger than the packed faces of the reconstructed frame of 360-degree video data;
- derive an extended reference frame from the extended faces;
- and
- encode a subsequent frame of 360-degree video data using an inter-prediction process and the derived extended reference frame.
6 more elements
Dependencies are inferred from the claim text.