System and Related Methods for Reducing the Resource Consumption of a Convolutional Neural Network

A computing system comprising:

• one or more processors;
• a machine-learned model comprising a convolutional neural network, the convolutional neural network comprising a plurality of convolutional layers comprising a plurality of kernels, the machine-learned model being configured to receive a model input, and, in response to receipt of the model input, output a model output;
• one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the computing system to perform operations, the operations comprising:
• obtaining data descriptive of the convolutional neural network, wherein the convolutional neural network comprises a plurality of convolutional layers configured to perform convolutions using a plurality of kernels, each of the plurality of kernels comprising a plurality of kernel elements;
• training, for one or more training iterations, the convolutional neural network using a loss function that comprises a group sparsifying regularizer term configured to sparsify a respective subset of the kernel elements of each of one or more kernels of the plurality of kernels of the convolutional neural network;
• following at least one training iteration, determining for each of the one or more kernels, whether to modify a respective size of such kernel to remove the respective subset of the kernel elements based at least in part on respective values of the respective subset of kernel elements associated with such kernel;
• and
• modifying the respective size of at least one of the one or more kernels to remove the respective subset of the kernel elements.

A computing system comprising:

• one or more processors;
• one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the computing system to perform operations, the operations comprising:
• receiving a machine-learned model comprising a convolutional neural network, wherein the convolutional neural network comprises a plurality of convolutional layers configured to perform convolutions using a plurality of kernels, each of the plurality of kernels comprising a plurality of kernel elements;
• determining, by the one or more computing devices, for at least one of the plurality of kernels, whether to modify a respective size of the at least one of the plurality of kernels to remove the respective subset of the kernel elements based at least in part on respective values of the respective subset of kernel elements associated with such kernel;
• and
• modifying, by the one or more computing devices, the respective size of at least one of the one or more kernels to remove the respective subset of the kernel elements.