Method and apparatus for video coding for machine vision

An apparatus for encoding video for machine vision and human/machine hybrid vision, the apparatus comprising:

• at least one memory configured to store computer program code;
• at least one processor configured to access the computer program code and operate as instructed by the computer program code, the computer program code including:
• setting code configured to cause the at least one processor to receive at a hybrid codec, input including at least one of video or image data, the hybrid codec including a first and second codec, wherein the first codec is a traditional codec designed for human consumption and the second codec is a learning-based codec designed for machine vision;
• compressing code configured to cause the at least one processor to compress the input using the first codec, wherein the compressing code includes down-sampling code configured to cause the at least one processor to down-sample the input using a down-sampling module, and the compressing code includes up-sampling code configured to cause the at least one processor to up-sample the compressed input using an up-sampling module producing a residual signal;
• quantizing code configured to cause the at least one processor to quantize the residual signal to obtain a quantized representation of the input;
• entropy encoding code configured to cause the at least one processor to encode the quantized representation of the input using one or more convolutional filter modules;
• and
• training code configured to cause the one or more networks using the entropy encoded quantized representation.

A non-transitory computer readable medium having stored thereon computer instructions that when executed by at least one processor cause the at least one processor to:

• receive at a hybrid codec, input including at least one of video or image data, the hybrid codec including a first and second codec, wherein the first codec is a traditional codec designed for human consumption and the second codec is a learning-based codec designed for machine vision;
• compress the input using the first codec, wherein the compressing includes down-sampling the input using a down-sampling module and up-sampling the compressed input using an up-sampling module producing a residual signal;
• quantize the residual signal to obtain a quantized representation of the input;
• entropy encode the quantized representation of the input using one or more convolutional filter modules;
• and
• train one or more networks using the entropy encoded quantized representation.