Systems and methods for content adaptive multi-scale feature layer filtering and redundant channel processing

A decoder in a system for video coding for machines, comprising circuitry configured to:

• receive an encoded bitstream generated by an encoder that generates a plurality of feature layers and selectively filters the plurality of plurality of feature layers prior to encoding, the bitstream comprising the filtered feature layers and signaling information identifying which layers were impacted by filtering;
• decompress the encoded bitstream;
• applying the signaling information to the decompressed bitstream and generating layers and channels removed by filtering at the encoder;
• applying the reconstructed feature layers and channels to a neural network trained for a machine task.

A method of transmitting an encoded bitstream for video coding for machines, comprising:

• applying a neural network front end, the neural network front end receiving image data and generating a plurality of feature layers;
• applying a layer context processor evaluating context of objects in the image data which impacts the significance of the layers of the feature map for a machine task;
• applying a redundant layer identifier, the redundant layer identifier applying the output of the context processor and determining the relevance of the plurality of feature layers to a machine task;
• applying a layer filter, the layer filter receiving the plurality of feature layers from the neural network front end and the output of the redundant layer identifier and performing at least one of removing a redundant layer and scaling a layer identified as having low relevance to a machine task to generate a filtered layer set;
• applying a redundant channel identifier, the redundant channel identifier determining a correlation among a plurality of channels in at least one feature layer a channel filter selecting one of a plurality of correlated channels to encode and removing the remaining plurality of correlated channels;
• and
• applying an encoder receiving the filtered layer set and channels and generating the coded bitstream for transmission.