System and method for automated design space determination for deep neural networks

A non-transitory computer readable medium comprising computer executable instructions for automated design space exploration for deep neural networks, the computer executable instructions comprising instructions for:

• obtaining a teacher model deep neural network;
• obtaining one or more constraints, the one or more constraints being associated with at least one of:
• a computer-implemented application, a target computing device or a process used in the computer-implemented application, the computer-implemented application configured to utilize a deep neural network having multiple layers with associated weights and computational parameters;
• learning an optimal deep neural network using the teacher model deep neural network, the one or more constraints, a training data set, and a validation data set, wherein learning the optimal deep neural network comprises iteratively:
• generating a new student deep neural network by applying network transformation operations to the teacher model deep neural network, wherein the network transformation operations comprise at least one of:
• altering a number of filters in a layer, altering a number of nodes in the layer, adding the layer to the teacher model deep neural network, or removing the layer from the teacher model deep neural network, and wherein generating the new student deep neural network further comprises applying function-preserving transformations and reusing weights from the teacher model to maintain functional equivalence prior to retraining with knowledge distillation;
• evaluating the new student deep neural network with a performance estimator, performance criteria, and the training data set, wherein the performance estimator measures at least one of:
• inference time in milliseconds, memory footprint in megabytes, or computational complexity measured in floating-point operations per second of the new student deep neural network, wherein evaluating the new student deep neural network with the performance estimator comprises estimating, in less than one second, performance on the target hardware so as to reduce computational overhead relative to full model retraining;
• in response to the new student deep neural network failing the performance criteria, bypassing evaluation with the validation data set;
• in response to the new student deep neural network satisfying the performance criteria, designating the new student deep neural network as being a satisfactory new student deep neural network and evaluating a performance of a satisfactory new student deep neural network with the validation data set;
• and
• generating a reward based on the evaluated performance of the satisfactory new student deep neural network, the reward being based on limitations of the target computing device, wherein the reward comprises a compression reward term and an accuracy reward term that are computed to incentivize production of smaller deep neural networks without sacrificing functional accuracy, wherein the reward is further computed based on hardware constraints of the target computing device, and wherein portions of the deep neural network are automatically mapped onto hardware cores of the target computing device maximizing energy efficiency and execution time on the target computing device;
• wherein the iterations converge, as measured by the reward, on an acceptable iteration of the optimal deep neural network;
• and
• deploying the optimal deep neural network on the target computing device or utilizing the optimal deep neural network in the process for use in the computer-implemented application, wherein deploying comprises deploying on a resource-constrained computing device.

A deep neural network optimization engine configured to perform automated design space exploration for deep neural networks, the engine comprising a processor and memory, the memory comprising computer executable instructions for:

• obtaining a teacher model deep neural network;
• obtaining one or more constraints, the one or more constraints being associated with at least one of:
• a computer-implemented application, a target computing device, or a process used in the computer-implemented application, the computer-implemented application configured to utilize a deep neural network having multiple layers with associated weights and computational parameters;
• learning an optimal deep neural network using the teacher model deep neural network, the one or more constraints, a training data set, and a validation data set, wherein learning the optimal deep neural network comprises iteratively:
• generating a new student deep neural network, and wherein generating the new student deep neural network further comprises applying function-preserving transformations and reusing weights from the teacher model to maintain functional equivalence prior to retraining with knowledge distillation;
• evaluating the new student deep neural network with a performance estimator, performance criteria, and the training data set by applying network transformation operations to the teacher model deep neural network, wherein the performance estimator measures at least one of:
• inference time in milliseconds, memory footprint in megabytes, or computational complexity measured in floating-point operations per second of the new student deep neural network, wherein the network transformation operations comprise at least one of:
• altering a number of filters in a layer, altering a number of nodes in the layer, adding the layer to the teacher model deep neural network, or removing the layer from the teacher model deep neural network, wherein evaluating a new student deep neural network with the performance estimator comprises estimating performance on the target hardware so as to reduce computational overhead relative to full model retraining;
• in response to the new student deep neural network failing the performance criteria, bypassing evaluation with the validation data set;
• in response to the new student deep neural network satisfying the performance criteria, designating the new student deep neural network as being a satisfactory new student deep neural network and evaluating a performance of a satisfactory new student deep neural network with the validation data set;
• and
• generating a reward based on the evaluated performance of the satisfactory new student deep neural network, the reward being based on limitations of the target computing device, wherein the reward comprises a compression reward term and an accuracy reward term that are computed to incentivize production of smaller deep neural networks without sacrificing functional accuracy, wherein the reward is further computed based on hardware constraints of the target computing device, and wherein portions of the deep neural network are automatically mapped onto hardware cores of the target computing device maximizing energy efficiency and execution time on the target computing device;
• wherein the iterations converge, as measured by the reward, on an acceptable iteration of the optimal deep neural network;
• and
• deploying the optimal deep neural network on the target computing device or utilizing the optimal deep neural network in the process for use in the computer-implemented application, wherein deploying comprises deploying on are source-constrained computing device.