Substitutional end-to-end video coding
AssigneeTencent America LLC
InventorsXiao Wang, Wei Jiang, Wei Wang, Shan Liu
Life of this patent
Estimating the term…
Abstract
A method of video encoding at a video encoder can include receiving a picture, and determining a substitutional representation of the picture by performing an optimization process to tune elements of the substitutional representation of the picture to optimize a rate-distortion performance of coding the substitutional representation of the picture based on an end-to-end (E2E) optimized framework. The E2E optimized framework can be an artificial neural network (ANN) based video coding framework that is pretrained. The substitutional representation of the picture can be encoded to generate a bitstream.
Claims18
A method of video encoding at a video encoder, comprising:
- receiving a picture;
- determining a substitutional picture of the picture by performing an optimization process to tune elements of the substitutional picture of the picture to optimize a rate-distortion performance of coding the substitutional picture of the picture based on an end-to-end (E2E) optimized framework, the E2E optimized framework being an artificial neural network (ANN) based video coding framework that is pretrained, the optimization process being performed in an iterative manner based on the E2E optimized frame work to optimize a loss function with respect to the elements of the substitutional picture of the picture;
- and
- encoding the substitutional picture of the picture to generate a bitstream, wherein
- the loss function indicates the rate-distortion performance of coding the substitutional picture of the picture based on the E2E optimized framework, and
- during the optimization process that is performed in the iterative manner, values of the elements of the substitutional picture of the picture are tuned.
An apparatus of video encoding, comprising circuitry configured to:
- receive a picture;
- determine a substitutional picture of the picture by performing an optimization process to tune elements of the substitutional picture of the picture to optimize a rate-distortion performance of coding the substitutional picture of the picture based on an end-to-end (E2E) optimized framework, the E2E optimized framework being an artificial neural network (ANN) based video coding framework that is pretrained, the optimization process being performed in an iterative manner based on the E2E optimized frame work to optimize a loss function with respect to the elements of the substitutional picture of the picture;
- and
- encode the substitutional picture of the picture to generate a bitstream, wherein
- the loss function indicates the rate-distortion performance of coding the substitutional picture of the picture based on the E2E optimized framework, and
- during the optimization process that is performed in the iterative manner, values of the elements of the substitutional picture of the picture are tuned.
A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform a method of video encoding, the method comprising:
- receiving a picture;
- determining a substitutional picture of the picture by performing an optimization process to tune elements of the substitutional picture of the picture to optimize a rate-distortion performance of coding the substitutional picture of the picture based on an end-to-end (E2E) optimized framework, the E2E optimized framework being an artificial neural network (ANN) based video coding framework that is pretrained, the optimization process being performed in an iterative manner based on the E2E optimized frame work to optimize a loss function with respect to the elements of the substitutional picture of the picture;
- and
- encoding the substitutional picture of the picture to generate a bitstream, wherein
- the loss function indicates the rate-distortion performance of coding the substitutional picture of the picture based on the E2E optimized framework, and
- during the optimization process that is performed in the iterative manner, values of the elements of the substitutional picture of the picture are tuned.
Dependencies are inferred from the claim text.