US 11,636,333 B2 · Optimizing neural network structures for embedded systems

Abstract

A model training and implementation pipeline trains models for individual embedded systems. The pipeline iterates through multiple models and estimates the performance of the models. During a model generation stage, the pipeline translates the description of the model together with the model parameters into an intermediate representation in a language that is compatible with a virtual machine. The intermediate representation is agnostic or independent to the configuration of the target platform. During a model performance estimation stage, the pipeline evaluates the performance of the models without training the models. Based on the analysis of the performance of the untrained models, a subset of models is selected. The selected models are then trained and the performance of the trained models are analyzed. Based on the analysis of the performance of the trained models, a single model is selected for deployment to the target platform.

Claims24

Dependent claims

A method for generating a machine-learned model comprising:

generating an untrained model;
generating an intermediate representation of the untrained model, the intermediate representation in an intermediate language compatible with a virtual machine;
evaluating the performance of the untrained model, wherein evaluating the performance includes at least one of determining a latency in applying the untrained model in a target system, determining a frequency at which the untrained model can be applied in the target system, determining an amount of resources used by the untrained model, and determining an amount of power consumed by the target system using the untrained model;
iteratively generating and evaluating new untrained models, wherein individual new untrained models are generated based on performance of one or more previous untrained models;
selecting a subset of models based on the performance of the generated models;
training the selected subset of models;
evaluating respective accuracies of the subset of models;
and
selecting a particular model of the subset of models for deployment to the target system based on the accuracies.

6 more elements

A system comprising one or more processors and non-transitory computer readable media storing instructions that when executed by the one or more processors, cause the one or more processors to:

generate an untrained model;
generate an intermediate representation of the untrained model, the intermediate representation in an intermediate language compatible with a virtual machine;
evaluate the performance of the untrained model, wherein evaluating the performance includes at least one of determining a latency in applying the untrained model in a target system, determining a frequency at which the untrained model can be applied in the target system, determining an amount of resources used by the untrained model, and determining an amount of power consumed by the target system using the untrained model;
iteratively generate and evaluate new untrained models, wherein individual new untrained models are generated based on performance of one or more previous untrained models;
select a subset of models based on the performance of the generated models;
train the selected subset of models;
evaluate respective accuracies of the subset of models;
and
select a particular model of the subset of models for deployment to the target system based on the accuracies.

6 more elements

Non-transitory computer readable media storing instructions that when executed by a system of one or more processors, cause the one or more processors to:

generate an untrained model;
generate an intermediate representation of the untrained model, the intermediate representation in an intermediate language compatible with a virtual machine;
evaluate the performance of the untrained model, wherein evaluating the performance includes at least one of determining a latency in applying the untrained model in a target system, determining a frequency at which the untrained model can be applied in the target system, determining an amount of resources used by the untrained model, and determining an amount of power consumed by the target system using the untrained model;
iteratively generate and evaluate new untrained models, wherein individual new untrained models are generated based on performance of one or more previous untrained models;
select a subset of models based on the performance of the generated models;
train the selected subset of models;
evaluate respective accuracies of the subset of models;
and
select a particular model of the subset of models for deployment to the target system based on the accuracies.

6 more elements

Dependencies are inferred from the claim text.

Optimizing neural network structures for embedded systems

Life of this patent

Optimizing neural network structures for embedded systems

Life of this patent

Abstract

Claims24