Generic modular sparse three-dimensional (3D) convolution design utilizing sparse 3D group convolution

A method for sparse three-dimensional (3D) convolution acceleration as part of training a neural network model implemented on a data processing system to perform analysis or processing of image data, the method comprising:

• receiving, by a first convolutional layer of a plurality of convolutional layers of the neural network model an input feature map derived from the image data;
• linearly increasing sparsity of the input feature map, by partitioning, by the first convolutional layer, the input feature map into a plurality (G) of disjoint input groups having dimensions of magnitudes equal to the input feature map;
• generating, by the first convolutional layer, a plurality of disjoint output groups corresponding to the plurality of disjoint input groups by, for each input group of the plurality of disjoint input groups, performing, a convolution operation using a shared 3D convolutional kernel/filter associated with the first convolutional layer on all input feature values of all or a subset of active/valid voxels of the input group to produce corresponding output feature values within a corresponding output group of the plurality of disjoint output groups by sliding the convolutional kernel/filter according to a stride, left-to-right and top-to-bottom, within a spatial dimension of the input group;
• and
• creating, by the first convolutional layer, an output feature map with enhanced feature representation for use by a subsequent convolutional layer of the plurality of convolutional layers, wherein the output feature map merges all output feature values associated with active/valid voxels of each of the plurality of disjoint output groups into the output feature map by sequentially stacking the plurality of disjoint output groups.

A non-transitory computer-readable storage medium embodying a set of instructions, which when executed by one or more processors, including a graphics processor, causes the one or more processors to, as part of training of a neural network model to perform analysis or processing of image data:

• receive, by a first convolutional layer of a plurality of convolutional layers of the neural network model, an input feature map derived from the image data;
• linearly increase sparsity of the input feature map, by partitioning, by the first convolutional layer, the input feature map into a plurality (G) of disjoint input groups having dimensions of magnitudes equal to the input feature map;
• generate, by the first convolutional layer, a plurality of disjoint output groups corresponding to the plurality of disjoint input groups by, for each input group of the plurality of disjoint input groups, performing, a convolution operation using a shared 3D convolutional kernel/filter associated with the convolutional layer on all input feature values of all or a subset of active/valid voxels of the input group to produce corresponding output feature values within a corresponding output group of the plurality of disjoint output groups by sliding the shared 3D convolutional kernel/filter according to a stride, left-to-right and top-to-bottom, within a spatial dimension of the input group;
• and
• create, by the first convolutional layer, an output feature map with enhanced feature representation for use by a subsequent convolutional layer of the plurality of convolutional layers, wherein the output feature map merges all output feature values associated with active/valid voxels of each of the plurality of disjoint output groups into the output feature map by sequentially stacking the plurality of disjoint output groups.