Scene-adaptive online learning for video post processing

An apparatus configured to process video data at a capture device, the apparatus comprising:

• a memory;
• and
• processing circuitry in communication with the memory, the processing circuitry configured to:
• encode a set of original pictures to create encoded video data;
• decode the encoded video data to create a set of reconstructed pictures;
• determine a subset of parameters to update from among a plurality of parameters of a post-processing filter network, including using the set of original pictures as ground truth and the set of reconstructed pictures as input to the post-processing filter network, wherein to determine the subset of parameters to update, the processing circuitry is further configured to:
• accumulate a respective gradient change for every parameter of the plurality of parameters over N forward and backward passes;
• sort the respective gradient changes in a list;
• and
• select parameters associated with a top ‘n’ percent of the respective gradient changes in the list as the subset of parameters;
• update the subset of parameters to generate updated parameters;
• and
• send the encoded video data and the updated parameters to a playback device.

A method of processing video data at a playback device, the method comprising:

• receiving encoded video data and encoded parameters for a post-processing filter network, wherein the encoded parameters include an arithmetic parameter ‘n’, an arithmetically encoded mask, and a respective parameter difference for each of the updated parameters;
• decoding the encoded video data to generate a decoded picture;
• decoding the encoded parameters to recover updated parameters, wherein decoding the encoded parameters comprises:
• decoding the arithmetic parameter ‘n’;
• arithmetically decoding the arithmetically encoded mask using arithmetic parameter ‘n’ to recover a mask, wherein each value of the mask indicates whether or not a parameter in the post-processing filter network is the updated parameter;
• and
• adding the respective parameter difference to a corresponding parameter value of the post-processing filter network based on the mask to recover the updated parameters;
• and;
• processing the decoded picture using the post-processing filter network and the updated parameters.

An apparatus configured to process video data at a playback device, the apparatus comprising:

• a memory;
• and
• processing circuitry in communication with the memory, the processing circuitry configured to:
• receive encoded video data and encoded parameters for a post-processing filter network, wherein the encoded parameters include an arithmetic parameter ‘n’, an arithmetically encoded mask, and a respective parameter difference for each of the updated parameters;
• decode the encoded video data to generate a decoded picture;
• decode the encoded parameters to recover updated parameters, wherein to decode the encoded parameters, the processing circuitry is further configured to:
• decode the arithmetic parameter ‘n’;
• arithmetically decode the arithmetically encoded mask using arithmetic parameter ‘n’ to recover a mask, wherein each value of the mask indicates whether or not a parameter in the post-processing filter network is the updated parameter;
• and
• add the respective parameter difference to a corresponding parameter value of the post-processing filter network based on the mask to recover the updated parameters;
• and
• process the decoded picture using the post-processing filter network and the updated parameters to generate a filtered picture.