Apparatus, a method and a computer program for video coding and decoding

A method comprising:

• obtaining an encoded block of a picture or an encoded picture in a decoder;
• determining when the block or picture is used for on-line learning;
• when affirmative, decoding the block or picture;
• reconstructing a coarse version of the block or picture or a respective prediction error block or picture;
• enhancing the coarse version using a neural net;
• and
• fine-tuning the neural net with a training signal based at least on the coarse version;
• decoding, responsive to the block or picture not being used for on-line learning, the block or picture;
• wherein reconstructing the respective prediction error block or picture forms a reconstructed prediction error block or picture;
• wherein decoding the block or picture comprises predicting the block or picture to form a prediction block or picture;
• and
• using the neural net for enhancing the reconstructed prediction error block or picture prior to reconstructing the block or picture with summing up the respective prediction block or picture and the respective prediction error block or picture.

An apparatus comprising:

• at least one processor;
• and
• at least one non-transitory memory including computer program code;
• the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus at least to perform:
• obtain a block of a picture or a picture in an encoder;
• determine when the block or picture is used for on-line learning;
• when affirmative, encode the block or picture;
• reconstruct a coarse version of the block or picture or a respective prediction error block or picture;
• enhance the coarse version using a neural net;
• and
• fine-tune the neural net with a training signal based at least on the coarse version;
• encode, responsive to the block or picture not being used for on-line learning, the block or picture;
• wherein reconstructing the respective prediction error block or picture forms a reconstructed prediction error block or picture;
• wherein encoding the block or picture comprises predicting the block or picture to form a prediction block or picture;
• and
• use the neural net for enhancing the reconstructed prediction error block or picture prior to reconstructing the block or picture with summing up the respective prediction block or picture and the respective prediction error block or picture.

An apparatus comprising:

• at least one processor;
• and
• at least one non-transitory memory including computer program code;
• the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus at least to perform:
• obtain an encoded block of a picture or an encoded picture in a decoder;
• determine when the block or picture is used for on-line learning;
• when affirmative, decode the block or picture;
• reconstruct a coarse version of the block or picture or the respective prediction error block or picture;
• enhance the coarse version using a neural net;
• and
• fine-tune the neural net with a training signal based at least on the coarse version;
• decode, responsive to the block or picture not being used for on-line learning, the block or picture;
• wherein reconstructing the respective prediction error block or picture forms a reconstructed prediction error block or picture;
• wherein decoding the block or picture comprises predicting the block or picture to form a prediction block or picture;
• and
• use the neural net for enhancing the reconstructed prediction error block or picture prior to reconstructing the block or picture with summing up the respective prediction block or picture and the respective prediction error block or picture.

A non-transitory program storage device readable by a machine, tangibly embodying a program of instructions executable with the machine for performing operations, the operations comprising:

• obtaining a block of a picture or a picture in an encoder;
• determining when the block or picture is used for on-line learning;
• when affirmative, encoding the block or picture;
• reconstructing a coarse version of the block or picture or a respective prediction error block or picture;
• enhancing the coarse version using a neural net;
• and
• fine-tuning the neural net with a training signal based at least on the coarse version;
• encoding, responsive to the block or picture not being used for on-line learning, the block or picture;
• wherein reconstructing the respective prediction error block or picture forms a reconstructed prediction error block or picture;
• wherein encoding the block or picture comprises predicting the block or picture to form a prediction block or picture;
• and
• using the neural net for enhancing the reconstructed prediction error block or picture prior to reconstructing the block or picture with summing up the respective prediction block or picture and the respective prediction error block or picture.

A non-transitory program storage device readable by a machine, tangibly embodying a program of instructions executable with the machine for performing operations, the operations comprising:

• obtaining an encoded block of a picture or an encoded picture in a decoder;
• determining when the block or picture is used for on-line learning;
• when affirmative, decoding the block or picture;
• reconstructing a coarse version of the block or picture or a respective prediction error block or picture;
• enhancing the coarse version using a neural net;
• and
• fine-tuning the neural net with a training signal based at least on the coarse version;
• decoding, responsive to the block or picture not being used for on-line learning, the block or picture;
• wherein reconstructing the respective prediction error block or picture forms a reconstructed prediction error block or picture;
• wherein decoding the block or picture comprises predicting the block or picture to form a prediction block or picture;
• and
• using the neural net for enhancing the reconstructed prediction error block or picture prior to reconstructing the block or picture with summing up the respective prediction block or picture and the respective prediction error block or picture.