Receptive-Field-Conforming Convolutional Models for Video Coding

A method of determining a block partitioning in video coding using a convolutional neural network (CNN), the block having an N×N size and a smallest partition determined by the CNN being of size S×S, comprising:

• extracting, using feature extraction layers of the CNN, first feature maps of the block, wherein each first feature map is of size S×S;
• concatenating, using a concatenation layer of the CNN, the first feature maps of the block;
• and
• inferring, by classifiers of the CNN each including classification layers, block partitioning, by:
• receiving, by each classification layer, second feature maps having a respective feature dimension;
• inferring, by the each classifier, partition decisions for sub-blocks of size (αS)×(αS) of the block, wherein α is a power of 2 and α=2, . . . , N/S, by:
• applying, at some of successive classification layers of the classification layers, a kernel of size 1×1 to reduce the respective feature dimension in half;
• and
• outputting by a last layer of the classification layers an output corresponding to a N/(αS)×N/(αS)×1 output map.

An apparatus for decoding an image block, comprising:

• a processor that:
• receives, in a compressed bitstream, an indication of a quad-tree partitioning of the image block into sub-blocks, wherein an encoder determined the quad-tree partitioning of the image block using a convolutional neural network (CNN) that includes:
• feature extraction layers;
• a concatenation layer that receives, from the feature extraction layers, first feature maps of the block, wherein each first feature map is of size S×S;
• and
• classifiers,
• each classifier comprising classification layers, wherein each classification layer receives second feature maps having a respective feature dimension,
• the each classifier is configured to infer partition decisions for sub-blocks of size (αS)×(αS) of the block, wherein α is a power of 2 and α=2, . . . , N/S, by:
• applying, at some of successive classification layers of the classification layers, a kernel of size 1×1 to reduce the respective feature dimension in half;
• and
• outputting by a last layer of the classification layers an output corresponding to a N/(αS)×N/(αS)×1 output map;
• and
• decodes the image block using the indication of the quad-tree partitioning of the image block.