Back to results
US 2024/0187640 A1Published application
Temporal structure-based conditional convolutional neural networks for video compression
AssigneeVid Scale, Inc.
InventorsFabien RACAPE, Jean BEGAINT, Simon FELTMAN, Akshay PUSHPARAJA
Life of this patent
Estimating the term…
Abstract
Video encoding and decoding is implemented with auto encoders using luminance information to derive motion information for chrominance prediction. In one embodiment conditional convolutions are used to encode motion flow information. A current condition, for example, GOP structure, is used as input to a succession of fully connected layers to implement the conditional convolution. In a related embodiment, more than one reference frame is used to encode motion flow information.
Claims21
A method, comprising:
- receiving an input to a conditional convolution layer, said input comprising a concatenated tensor of a current and at least one reference block;
- performing at least one conditional convolution on said input based on data representative of a current condition, said conditional convolution comprising a series of fully connected layers;
- encoding motion flow using an output from the at least one conditional convolution, wherein the current condition is based on GOP structure;
- and,
- generating a bitstream comprising said encoded motion flow.
2 more elements
An apparatus, comprising:
- a processor, configured to:
- receive an input to a conditional convolution layer, said input comprising a concatenated tensor of a current and at least one reference block;
- perform at least one conditional convolution on said input based on data representative of a current condition, said conditional convolution comprising a series of fully connected layers;
- encode motion flow using an output from the at least one conditional convolution, wherein the current condition is based on GOP structure;
- and,
- generating a bitstream comprising said encoded motion flow.
3 more elements
A method, comprising:
- entropy decoding a bitstream comprising motion flow data of a current block of video data;
- performing at least one conditional deconvolution on said bitstream to generate a reconstructed residue, said conditional deconvolution comprising a series of fully connected layers;
- and,
- combining said reconstructed residue with a prediction of said current block to generate a decoded block.
1 more element
An apparatus, comprising:
- a processor, configured to:
- entropy decode a bitstream comprising motion flow data of a current block of video data;
- perform at least one conditional deconvolution on said bitstream to generate a reconstructed residue, said conditional deconvolution comprising a series of fully connected layers;
- and,
- combine said reconstructed residue with a prediction of said current block to generate a decoded block.
2 more elements
(canceled)
Dependencies are inferred from the claim text.