Optimizing machine learning models
AssigneeOctoML, Inc.
InventorsMatthew Welsh, Jason Knight, Jared Roesch, Thierry Moreau, Adelbert Chang, Tianqi Chen, Luis Henrique Ceze, An Wang, Michal Piszczek, Andrew McHarg, Fletcher Haynes
Life of this patent
Estimating the term…
Abstract
A facility for optimizing machine learning models is described. The facility obtains a description of a machine learning model and a hardware target for the machine learning model. The facility obtains optimization result data from a repository of optimization result data. The facility optimizes the machine learning model for the hardware target based on the optimization result data.
Claims19
One or more instances of computer-readable media collectively having contents configured to cause a computing device to obtain a plurality of machine learning models and optimize the plurality of machine learning models, none of the instances of computer-readable media constituting a transitory propagating signal per se, the method comprising:
- obtaining a plurality of machine learning models, including obtaining a description of each machine learning model of the plurality of machine learning models;
- and
- for each machine learning model of the plurality of machine learning models:
- identifying a hardware target of the machine learning model, wherein the hardware target includes an indication of hardware upon which the machine learning model is to be deployed;
- identifying a model type of the machine learning model based on the description of the machine learning model;
- retrieving optimization result data from a repository of optimization result data based on the identified hardware target and the model type, the retrieved optimization result data reflecting a level of consumption of hardware resources resulting from changing one or more machine learning models;
- changing the machine learning model for optimized operation on hardware specified by the identified hardware target, based on the hardware target and the retrieved optimization result data, wherein changing the machine learning model comprises changing one or more of:
- software code associated with one or more operators used by the machine learning model;
- or
- software code associated with one or more partitions of the machine learning model;
- obtaining additional optimization result data by evaluating the performance of the optimized machine learning model on hardware indicated by the hardware target, the additional optimization result data including an indication of the hardware target and the model type;
- and
- storing the additional optimization result data within the repository of optimization result data.
A system to obtain a plurality of machine learning models and optimize the plurality of machine learning models, the system comprising:
- an optimization result data repository configured to store optimization logs containing optimization result data for machine learning models;
- and
- a computing device configured to:
- obtain an indication of one or more machine learning models and one or more model types each corresponding to a machine learning model of the one or more machine learning models;
- and
- for each machine learning model:
- identify a hardware target of the machine learning model, wherein the hardware target includes an indication of hardware upon which the machine learning model is to be deployed;
- retrieve optimization result data from the repository based on the identified hardware target and the model type that corresponds to the machine learning model, the retrieved optimization result data reflecting a level of consumption of hardware resources resulting from changing one or more machine learning models;
- changing software code used to implement the machine learning model for optimized operation on hardware specified by the identified hardware target, based on the hardware target and the retrieved optimization result data;
- obtain second optimization result data by evaluating the performance of the optimized machine learning model on hardware indicated by the identified hardware target, the second optimization result data including an indication of the hardware target and the model type;
- and
- store the second optimization result data in the optimization result data repository.
One or more storage devices collectively storing a machine learning model retrieval and optimization result data structure, none of the storage devices constituting a transitory propagating signal per se, the data structure comprising:
- information specifying a machine learning model, including a model type of the machine learning model;
- information specifying a hardware target of the machine learning model, the information specifying the hardware target including an indication of hardware upon which the machine learning model is to be deployed;
- and
- information specifying optimization result data, the optimization result data being associated with one or more hardware targets and one or more model types wherein the optimization result data reflects a level of consumption of hardware resources resulting from changing one or more machine learning models;
- information specifying second optimization result data,
- such that, the information specifying optimization result data and the hardware target are used to change software code that defines the machine learning model, and
- the information specifying the second optimization result data is obtained by evaluating the performance of the changed machine learning model on hardware specified by the hardware target.
Dependencies are inferred from the claim text.