Method and system for two-step hierarchical model optimization

A system for model optimization, the system comprising:

• a memory storing instructions;
• one or more Input/Output (I/O) interfaces;
• and
• one or more hardware processors coupled to the memory via the one or more I/O interfaces, wherein the one or more hardware processors are configured by the instructions to:
• receive an input wherein the input further comprising a) a Deep learning (DL) model to be optimized, b) a training dataset for the DL model comprising a training dataset, a validation data and a test data, a validation data and a test data, wherein the training dataset is used for training, testing, and validating an optimized model, and c) resource constraints of an end device on which the DL model is to be implemented, wherein the resource constraints of the end device include hardware constraints specified by an end user to ensure optimization is obtained for the DL model, retaining maximum accuracy, reducing loss due including small memory input;
• identify an appropriate technique for the received DL model for the hardware constraints specified by an end user to ensure best optimization is obtained for the DL model, retaining maximum accuracy, reducing loss due;
• apply the identified technique, a two-step hierarchical model optimization approach on the DL model to obtain an optimized model (Mopt) fitting to the resource constraints, by utilizing an Automated Pruning-weight Clustering-Quantization (AutoPCQ) technique followed by conditional application of an Automated Knowledge Distillation (AKD) technique, wherein the two-step hierarchical model optimization approach comprising:
• (A) triggering the AutoPCQ technique that formulates a problem of configuration selection of the DL model as an optimization problem by iteratively applying Bayesian Optimization (BO) and Reinforcement Learning (RL) automates the process to arrive at an optimal configuration using the AutoPCQ, wherein, the AutoPCQ technique comprising:
• (i) defining a Configuration Space (CS) for the DL model on which optimization is to be carried out, wherein the CS further comprising parameter configurations for a pruning (Cp), a weight clustering (Cwc), a quantization (Cq), and a model layer configuration (Cm) for the DL model, wherein {CS=Cp ∪CwcUCq ∪Cm}, provide a unified way to define configurations space and automatically generate configurations space with specific format;
• and
• (ii) obtaining the optimized model (Mopt) of the DL model in accordance with the Configuration Space {CS=Cp ∪Cwc∪Cq ∪Cm} using the training dataset by iteratively performing, for a preset number of trials, steps further comprising:
• 1) Retrieving one or more best configurations (c) from a database KSTORE for the DL model in accordance with the Configuration Space (CS) based on attribute matching with respect to a plurality of attributes derived for the DL model;
• 2) Performing an initial optimizing of the DL model to obtain a current Mopt based on the retrieved one or more configurations;
• 3) Evaluating the current Mopt using the trained dataset to determine a loss and a compression factor with respect to the DL model;
• and
• 4) Comparing the loss in each iteration against a preset loss (δ) to identify the current Mopt as the optimized model (Mopt) if the loss is less than a preset loss (δ), computed based on the resource constraints of the end device;
• optimizing the AutoPCQ technique for DL model optimization process, via pruning weight clustering quantization (PCQ), wherein the PCQ comprises:
• loading the DL a deep learning model and associated training data;
• constructing the CS a configuration space for the deep learning model;
• sampling one or more configurations from the CS configuration space;
• applying a sampled configuration to the deep learning model;
• training the DL deep learning model using the applied configuration;
• evaluating the trained model to determine an accuracy metric;
• and
• iteratively repeating the sampling, applying, training, and evaluating steps until a termination condition is met;
• and
• (B) automatically triggering the AKD technique if the loss for the current Mopt is greater than the preset loss (δ) after iterating through the preset number of trials, wherein the AKD technique formulates automated search of a student model as the optimization problem with the DL model representing a teacher model, wherein a search space for the student model is defined by a restricted Neural Network Architecture Search (rNAS) that restricts the search space based on (a) network attributes of the teacher model, (b) constraints on size reduction factor and (c) incorporating advance pruners for early stopping of training of an identified sampled network if progress is not as required satisfactory, and wherein the student model obtained provides the optimized model (Mopt).

One or more non-transitory machine-readable information storage mediums comprising one or more instructions which when executed by one or more hardware processors cause:

• receiving an input wherein the input further comprising a) a Deep learning (DL) model to be optimized, b) a training dataset for the DL model comprising a training data, a validation data and a test data and, wherein the training dataset is used for training, testing, and validating an optimized model, c) resource constraints of an end device on which the DL model is to be implemented, wherein the resource constraints of the end device include hardware constraints specified by an end user to ensure optimization is obtained for the DL model, retaining maximum accuracy, reducing loss due including small memory input;
• identifying an appropriate technique for the received DL model for the hardware constraints specified by an end user to ensure best optimization is obtained for the DL model, retaining maximum accuracy, reducing loss due;
• applying the identified technique, a two-step hierarchical model optimization approach on the DL model to obtain an optimized model (Mopt) fitting to the resource constraints, by utilizing an Automated Pruning-weight Clustering-Quantization (AutoPCQ) technique followed by conditional application of an Automated Knowledge Distillation (AKD) technique, wherein the two-step hierarchical model optimization approach comprising:
• triggering the AutoPCQ technique that formulates a problem of configuration selection of the DL model as an optimization problem by iteratively applying Bayesian Optimization (BO) and Reinforcement Learning (RL) automates the process to arrive at an optimal configuration using the AutoPCQ, wherein, the AutoPCQ technique comprising:
• (i) defining a Configuration Space (CS) for the DL model on which optimization is to be carried out, wherein the CS further comprising parameter configurations for a pruning (Cp), a weight clustering (Cwc), a quantization (Cq), and a model layer configuration (Cm) for the DL model, wherein {CS=Cp ∪CwcUCq ∪Cm}, provide a unified way to define configurations space and automatically generate configurations space with specific format;
• (ii) obtaining the optimized model (Mopt) of the DL model in accordance with the Configuration Space {CS=Cp ∪Cwc∪Cq ∪Cm} using the training dataset by iteratively performing, for a preset number of trials, steps further comprising:
• 1) Retrieving one or more best configurations (c) from a database KSTORE for the DL model in accordance with the Configuration Space (CS) based on attribute matching with respect to a plurality of attributes derived for the DL model;
• 2) Performing an initial optimizing of the DL model to obtain a current Mopt based on the retrieved one or more configurations;
• 3) Evaluating the current Mopt using the trained dataset to determine a loss and a compression factor with respect to the DL model;
• and
• 4) Comparing the loss in each iteration against a preset loss (δ) to identify the current Mopt as the optimized model (Mopt) if the loss is less than a preset loss (δ), computed based on the resource constraints of the end device;
• optimizing the AutoPCQ technique for DL model optimization process, via pruning weight clustering quantization (PCQ), wherein the PCQ comprises:
• loading the DL a deep learning model and associated training data;
• constructing the CS a configuration space for the DL deep learning model;
• sampling one or more configurations from the CS configuration space;
• applying a sampled configuration to the DL deep learning model;
• training the DL deep learning model using the applied configuration;
• evaluating the trained model to determine an accuracy metric;
• and
• iteratively repeating the sampling, applying, training, and evaluating steps until a termination condition is met;
• and
• (B) automatically triggering the AKD technique if the loss for the current Mopt is greater than the preset loss (δ) after iterating through the preset number of trials, wherein the AKD technique formulates automated search of a student model as the optimization problem with the DL model representing a teacher model, wherein a search space for the student model is defined by a restricted Neural Network Architecture Search (rNAS) that restricts the search space based on (a) network attributes of the teacher model, (b) constraints on size reduction factor and (c) incorporating advance pruners for early stopping of training of an identified sampled network if progress is not as required satisfactory, and wherein the student model obtained provides the optimized model (Mopt).