Automated processing of multiple prediction generation including model tuning
AssigneeDatabricks, Inc.
InventorsBenjamin Thomas Wilson, Corey Zumar
Life of this patent
Estimating the term…
Abstract
The present application discloses a method, system, and computer system for building a model associated with a dataset. The method includes receiving a data set, the dataset comprising a plurality of keys and a plurality of key-value relationships, determining a plurality of models to build based at least in part on the dataset, wherein determining the plurality of models to build comprises using the dataset format information to identify the plurality of models, building the plurality of models, and optimizing at least one of the plurality of models.
Claims19
A system, comprising:
- one or more processors configured to:
- receive a dataset, the dataset comprising a plurality of keys and a plurality of key-value relationships, wherein a key of the dataset corresponds to a grouping of columns of the dataset;
- determine a plurality of models to build based at least in part on the dataset, wherein determining the plurality of models to build comprises using a dataset format information to identify the plurality of models, wherein the determining of the plurality of models comprises to:
- analyze the dataset;
- extract a subset of the plurality of keys, wherein the subset of the plurality of keys includes a set of all unique combinations of keys of the dataset;
- and
- determine the plurality of models using the subset of the plurality of keys, wherein each unique combination of keys of the dataset corresponds to a model;
- analyze the dataset for which the corresponding model is built based on a unique dimension along which the dataset is analyzed for the each unique combination of keys for the dataset;
- build the plurality of models, comprising to:
- provide the dataset to train the plurality of models in parallel, one model per each grouping of dataset;
- and
- provide a unified API to interface with each model of the plurality of models;
- optimize at least one of the plurality of models;
- receive, from a user, a request for a prediction using the unified API, the request for the prediction comprising one or more prediction parameters;
- determine a prediction model based at least in part on the one or more prediction parameters, the prediction model being selected from among the plurality of models, wherein the determining of the prediction model comprises:
- use the prediction model to generate the prediction;
- and
- provide the prediction to the user;
- and
- a memory coupled to the one or more processors and configured to provide the one or more processors with instructions.
A method, comprising:
- receiving, by one or more processors, a data set, the dataset comprising a plurality of keys and a plurality of key-value relationships, wherein a key of the dataset corresponds to a grouping of columns of the dataset;
- determining a plurality of models to build based at least in part on the dataset, wherein determining the plurality of models to build comprises using a dataset format information to identify the plurality of models, wherein the determining of the plurality of models comprises:
- analyzing the dataset;
- extracting a subset of the plurality of keys, wherein the subset of the plurality of keys includes a set of all unique combinations of keys of the dataset;
- and
- determining the plurality of models using the subset of the plurality of keys, wherein each unique combination of keys of the dataset corresponds to a model;
- analyzing the dataset for which the corresponding model is built based on a unique dimension along which the dataset is analyzed for the each unique combination of keys for the dataset;
- building the plurality of models, comprising:
- providing the dataset to train the plurality of models in parallel, one model per each grouping of dataset;
- and
- providing a unified API to interface with each model of the plurality of models;
- optimizing at least one of the plurality of models;
- receiving, from a user, a request for a prediction using the unified API, the request for the prediction comprising one or more prediction parameters;
- determining a prediction model based at least in part on the one or more prediction parameters, the prediction model being selected from among the plurality of models, wherein the determining of the prediction model comprises:
- using the prediction model to generate the prediction;
- and
- providing the prediction to the user.
A computer program product embodied in a non-transitory computer readable medium and comprising computer instructions for:
- receiving, by one or more processors, a data set, the dataset comprising a plurality of keys and a plurality of key-value relationships, wherein a key of the dataset corresponds to a grouping of columns of the dataset;
- determining a plurality of models to build based at least in part on the dataset, wherein determining the plurality of models to build comprises using a dataset format information to identify the plurality of models, wherein the determining of the plurality of models comprises:
- analyzing the dataset;
- extracting a subset of the plurality of keys, wherein the subset of the plurality of keys includes a set of all unique combinations of keys of the dataset;
- and
- determining the plurality of models using the subset of the plurality of keys, wherein each unique combinations of keys of the dataset corresponds to a model;
- analyzing the dataset for which the corresponding model is built based on a unique dimension along which the dataset is analyzed for the each unique combination of keys for the dataset;
- building the plurality of models, comprising:
- providing the dataset to train the plurality of models in parallel, one model per each grouping of dataset;
- and
- providing a unified API to interface with each model of the plurality of models;
- optimizing at least one of the plurality of models,
- receiving, from a user, a request for a prediction using the unified API, the request for the prediction comprising one or more prediction parameters;
- determining a prediction model based at least in part on the one or more prediction parameters, the prediction model being selected from among the plurality of models, wherein the determining of the prediction model comprises:
- using the prediction model to generate the prediction;
- and
- providing the prediction to the user.
Dependencies are inferred from the claim text.