Learned latent residual coding of residual

A method comprising:

• determining, with a codec, a first latent tensor based, at least partially, on an input data item using, at least, a first set of layers;
• encoding the first latent tensor in a bitstream;
• decoding the encoded first latent tensor from the bitstream;
• determining a reconstructed data item based, at least partially, on the decoded first latent tensor using, at least, a second set of layers;
• determining residual information based, at least partially, on at least one of the input data item or information associated with the input data item, and at least one of the reconstructed data item or information associated with the reconstructed data item;
• determining a second latent tensor based, at least partially, on the residual information using, at least, a third set of layers;
• encoding the second latent tensor in the bitstream;
• decoding the encoded second latent tensor from the bitstream;
• determining a reconstructed residual signal based, at least partially, on the decoded second latent tensor and one or more auxiliary inputs using, at least, a fourth set of layers;
• and
• determining a decoded data item based, at least partially, on the reconstructed residual signal and at least one of the reconstructed data item, or the information associated with the reconstructed data item.

An apparatus comprising:

• at least one processor;
• and
• at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to:
• determine residual information based, at least partially, on at least one of a current input data item or information associated with the current input data item, and at least one of a previously coded input data item or information associated with the previously coded input data item;
• determine a latent tensor based, at least partially, on the residual information using, at least, a first set of layers;
• encode the latent tensor in a bitstream;
• decode the encoded latent tensor from the bitstream;
• determine a reconstructed residual signal based, at least partially, on the decoded latent tensor and at least one of:
• a first set of features derived from the previously coded input data item,
• information associated with the first set of features derived from the previously coded input data item,
• a second set of features from which the previously coded input data item was extracted, or
• information associated with the second set of features from which the previously coded input data item was derived,
• using, at least, a second set of layers;
• and
• determine a decoded data item based, at least partially, on the reconstructed residual signal and at least one of the previously coded input data item or the information associated with the previously coded input data item.

An apparatus comprising:

• at least one processor;
• and
• at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to:
• decode an encoded first latent tensor, associated with an input data item, from a bitstream;
• determine a reconstructed data item based, at least partially, on the decoded first latent tensor using, at least, a first set of layers;
• decode an encoded second latent tensor, associated with the input data item, from the bitstream;
• determine a reconstructed residual signal based, at least partially, on the decoded second latent tensor and one or more auxiliary inputs associated with the input data item;
• and
• determine a decoded data item based, at least partially, on the reconstructed residual signal and at least one of the reconstructed data item, or information associated with the reconstructed data item.