Image/video compression with scalable latent representation
AssigneeInterDigital VC Holdings, Inc.
InventorsWei JIANG, Hyomin CHOI, Fabien RACAPE
Life of this patent
Estimating the term…
Abstract
At least a method and an apparatus are presented for efficiently encoding or decoding video, for example human-centric video content. For example, at least one embodiment using a scalable latent representation comprising a generic codebook-based representation and a low-quality latent representation of a video. According to another embodiment, the scalable latent representation further comprises a domain-adaptive codebook-based representation. Advantageously, such scalable latent representation provides, for content such as human-centric video, a domain-adaptive and task-adaptive video coding framework that can be flexibly configured to accommodate both human and machine consumption.
Claims22
A method, comprising:
- receiving a scalable latent representation comprising a generic codebook-based representation and a low-quality latent representation of a sequence of images;
- obtaining a reconstructed generic codebook-based feature representative of image data samples reconstructed from the generic codebook-based representation;
- decoding the low-quality latent representation to obtain a reconstructed low-quality image;
- applying to the reconstructed low-quality image, a neural network-based embedding feature processing to generate a low-quality feature representative of a feature of image data samples;
- and
- applying to the reconstructed generic codebook-based feature and to the low-quality feature, a neural network-based reconstruction processing to generate a reconstructed image adapted to a plurality of computer vision tasks including both machine consumption and human consumption.
An apparatus comprising a memory and one or more processors, wherein
- the one or more processors are configured to:
- receive a scalable latent representation comprising a generic codebook-based representation and a low-quality latent representation of a sequence of images;
- obtain a reconstructed generic codebook-based feature representative of image data samples reconstructed from the generic codebook-based representation;
- decode the low-quality latent representation to obtain a reconstructed low-quality image;
- apply to the reconstructed low-quality image, a neural network-based embedding feature processing to generate a low-quality feature representative of a feature of image data samples;
- and
- apply to the reconstructed generic codebook-based feature and to the low-quality feature, a neural network-based reconstruction processing to generate a reconstructed image adapted to a plurality of computer vision tasks including both machine consumption and human consumption.
A method, comprising:
- obtaining a sequence of images to encode;
- applying to the sequence of images, a neural network-based generic embedding feature processing to generate a generic feature representative of a generic feature of image data samples;
- obtaining a generic codebook-based representation based on the generic feature and on a generic codebook;
- downsampling an image of the sequence of images to obtain a low-quality image;
- encoding the low-quality image to obtain a low-quality latent representation;
- and
- associating the generic codebook-based representation and the low-quality latent representation of the sequence of images to form a scalable latent representation of the sequence of images adapted to a plurality of computer vision tasks including both machine consumption and human consumption.
(canceled)
An apparatus comprising a memory and one or more processors, wherein
- the one or more processors are configured to obtain a sequence of images to encode;
- apply to the sequence of images, a neural network-based generic embedding feature processing to generate a generic feature representative of a generic feature of image data samples;
- obtain a generic codebook-based representation based on the generic feature and on a generic codebook;
- downsample an image of the sequence of images to obtain a low-quality image;
- encode the low-quality image to obtain a low-quality latent representation;
- and
- associate the generic codebook-based representation and the low-quality latent representation of the sequence of images to form a scalable latent representation of the sequence of images adapted to a plurality of computer vision tasks including both machine consumption and human consumption.
22-24. (canceled)
Dependencies are inferred from the claim text.