Deep contextual video image compression
AssigneeMicrosoft Technology Licensing, LLC
InventorsJiahao Li, Bin Li, Yan Lu
Life of this patent
Estimating the term…
Abstract
According to implementations of the present disclosure, there is provided a context-based image coding solution. According to the solution, a reference image of a target image is obtained. A contextual feature representation is extracted from the reference image, the contextual feature representation characterizing contextual information associated with the target image. Conditional encoding or conditional decoding is performed on the target image based on the contextual feature representation. In this way, the enhancement of the performance is achieved in terms of the reconstruction quality and the compression efficiency.
Claims20
A method for context-based image coding comprising:
- obtaining a reference image of a target image;
- extracting, using a machine learning model, an initial contextual feature representation from the reference image by converting the reference image from a pixel domain to a feature domain;
- estimating motion vector information or optical flow between the reference image and the target image;
- warping the initial contextual feature representation using the estimated motion vector information or optical flow to generate a warped feature tensor;
- refining the warped feature tensor using a context-fine-tuning neural network to obtain a final contextual feature representation that characterizes contextual information associated with the target image;
- and
- performing conditional encoding or conditional decoding on the target image based on the final contextual feature representation.
An electronic device for context-based image coding, comprising:
- a processor;
- and
- a memory coupled to the processor and having instructions stored thereon, the instructions, when executed by the processor, causing the electronic device to perform actions comprising:
- obtaining a reference image of a target image;
- extracting, using a machine learning model, an initial contextual feature representation from the reference image by converting the reference image from a pixel domain to a feature domain;
- estimating motion vector information or optical flow between the reference image and the target image;
- warping the initial contextual feature representation using the estimated motion vector information or optical flow to generate a warped feature tensor;
- refining the warped feature tensor using a context-fine-tuning neural network to obtain a final contextual feature representation that characterizes contextual information associated with the target image;
- and
- performing conditional encoding or conditional decoding on the target image based on the final contextual feature representation.
At least one non-transitory machine-readable medium comprising instructions for context-based image coding that, when executed by at least one processor, cause the at least one processor to perform operations to:
- obtain a reference image of a target image;
- extract, using a machine learning model, an initial contextual feature representation from the reference image by converting the reference image from a pixel domain to a feature domain, the contextual feature representation characterizing contextual information associated with the target image;
- and
- estimate motion vector information or optical flow between the reference image and the target image;
- warp the initial contextual feature representation using the estimated motion vector information or optical flow to generate a warped feature tensor;
- refine the warped feature tensor using a context-fine-tuning neural network to obtain a final contextual feature representation that characterizes contextual information associated with the target image;
- and
- perform conditional encoding or conditional decoding on the target image based on the final contextual feature representation.
Dependencies are inferred from the claim text.