Automated processing of multiple prediction generation including model tuning
AssigneeDatabricks, Inc.
InventorsBenjamin Thomas Wilson, Corey Zumar
Life of this patent
Estimating the term…
Abstract
The present application discloses a method, system, and computer system for building a model associated with a dataset. The method includes receiving a data set, the dataset comprising a plurality of keys and a plurality of key-value relationships, determining a plurality of models to build based at least in part on the dataset, wherein determining the plurality of models to build comprises using the dataset format information to identify the plurality of models, building the plurality of models, and optimizing at least one of the plurality of models.
Claims20
A computer-implemented method, comprising:
- accessing a set of machine-learning models trained using a dataset, wherein the dataset comprises at least a set of keys and key-values for the set of keys, and wherein each machine-learning model is trained using a respective subset of the dataset corresponding to a respective set of key-values for the set of keys;
- receiving, via an interface, a current dataset that is an updated dataset;
- determining whether drift has occurred with respect to the dataset used to train the set of machine-learning models, comprising determining whether a difference between the current dataset and the dataset exceeds a drift threshold;
- responsive to a determination the drift has occurred and the difference exceeds the drift threshold, identifying one or more machine-learning models in the set of machine-learning models that were trained using a drifted portion of the dataset;
- tuning parameters of the one or more machine-learning models in the set of machine-learning models using the current dataset to generate another set of machine-learning models;
- for a selected machine-learning model that is not determined to have drift, obtaining an accuracy of the machine-learning model;
- responsive to a determination that the accuracy is below an accuracy threshold, iteratively updating parameters of the selected machine-learning model by using a set of optimizers that are threads, compute nodes, or processor cores until the accuracy of the machine-learning model exceeds the accuracy threshold,
- wherein the set of optimizers retain a history of the iterative updates and stops when a maximum iterations are reached, stops when there is no improvement in the accuracy, or the accuracy decreases;
- and
- exposing the another set of machine-learning models via an interface of a computing service.
A non-transitory computer-readable storage medium comprising stored instructions executable by a processor system, the instructions when executed causing the processor system to:
- access a set of machine-learning models trained using a dataset, wherein the dataset comprises at least a set of keys and key-values for the set of keys, and wherein each machine-learning model is trained using a respective subset of the dataset corresponding to a respective set of key-values for the set of keys;
- receive, via an interface, a current dataset that is an updated dataset;
- determine whether drift has occurred with respect to the dataset used to train the set of machine-learning models, comprising determining whether a difference between the current dataset and the dataset exceeds a drift threshold;
- responsive to a determination the drift has occurred and the difference exceeds the drift threshold, identify one or more machine-learning models in the set of machine-learning models that were trained using a drifted portion of the dataset;
- tune parameters of the one or more machine-learning models in the set of machine-learning models using the current dataset to generate another set of machine-learning models;
- for a selected machine-learning model that is not determined to have drift, obtain an accuracy of the machine-learning model;
- responsive to a determination that the accuracy is below an accuracy threshold, iteratively update parameters of the selected machine-learning model by using a set of optimizers that are threads, compute nodes, or processor cores until the accuracy of the machine-learning model exceeds the accuracy threshold,
- wherein the set of optimizers retain a history of the iterative updates and stops when a maximum iterations are reached, stops when there is no improvement in the accuracy, or the accuracy decreases;
- and
- expose the another set of machine-learning models via an interface of a computing service.
A computer system, comprising:
- a processor system;
- and
- a non-transitory computer-readable storage medium comprising stored instructions executable by a processor system, the instructions when executed causing the processor system to:
- access a set of machine-learning models trained using a dataset, wherein the dataset comprises at least a set of keys and key-values for the set of keys, and wherein each machine-learning model is trained using a respective subset of the dataset corresponding to a respective set of key-values for the set of keys;
- receive, via an interface, a current dataset that is an updated dataset;
- determine whether drift has occurred with respect to the dataset used to train the set of machine-learning models, comprising determining whether a difference between the current dataset and the dataset exceeds a drift threshold;
- responsive to a determination the drift has occurred and the difference exceeds the drift threshold, identify one or more machine-learning models in the set of machined-learning models that were trained using a drifted portion of the dataset;
- tune parameters of the one or more machine-learning models in the set of machine-learning models using the current dataset to generate another set of machine-learning models;
- for a selected machine-learning model that is not determined to have drift, obtain an accuracy of the machine-learning model;
- responsive to a determination the accuracy is below an accuracy threshold, iteratively update parameters of the selected machine-learning model by using a set of optimizers that are threads, compute nodes, or processor cores until the accuracy of the machine-learning model exceeds the accuracy threshold,
- wherein the set of optimizers retain a history of the iterative updates and stops when a maximum iterations are reached, stops when there is no improvement in the accuracy, or the accuracy decreases;
- and
- expose the another set of machine-learning models via an interface of a computing service.
Dependencies are inferred from the claim text.