Function creation for database execution of deep learning model
AssigneeDatabricks, Inc.
InventorsSue Ann Hong, Shi Xin, Timothee Hunter, Ali Ghodsi
Life of this patent
Estimating the term…
Abstract
A function creation method is disclosed. The method comprises defining one or more database function inputs, defining cluster processing information, defining a deep learning model, and defining one or more database function outputs. A database function is created based at least in part on the one or more database function inputs, the cluster set-up information, the deep learning model, and the one or more database function outputs. In some embodiments, the database function enables a non-technical user to utilize deep learning models.
Claims14
A system for function creation, comprising:
- an interface configured to:
- receive an indication to create a database function;
- and
- a processor configured to:
- define one or more database function inputs, wherein the one or more database function inputs includes a structured query language (SQL) input;
- define cluster processing information, wherein the cluster processing information includes a performance threshold, an indication to determine resource allocation or batch size of the database function through a static analysis test without executing the database function, an indication to perform a pretest upon execution of the database function to determine resource allocation or batch size, an indication to adjust the resource allocation or the batch size of the database function in the event that performance of the database function is below the performance threshold during execution of the database function, and one or more of the following:
- author specified settings, definition of the static analysis test, and/or a definition of the pretest to be run at execution of the database function;
- define a deep learning model, comprising to:
- specify a deep learning model object;
- specify a deep learning model file location and a deep learning model framework, wherein the deep learning model file location specifies where a set of model files associated with the deep learning model is stored, wherein the deep learning model framework comprises a model file format and execution engine type, wherein the deep learning model framework is determined based on a first deep learning model object or a second deep learning model object, wherein computation is performed based on a first deep learning model framework or a second deep learning model framework, and wherein input and output nodes of the deep learning model are determined based on the set of model files and connected to the one or more database function inputs and one or more database function outputs;
- define the one or more database function outputs, wherein the one or more database function outputs includes one or more of the following:
- a map, vector, and/or an image;
- and
- create the database function based at least in part on the one or more database function inputs, the cluster processing information, the deep learning model, and the one or more database function outputs, comprising to:
- determine application logic based on the deep learning model;
- determine whether the resource allocation or the batch size is indicated;
- in response to a determination that the resource allocation or the batch size is indicated, store the resource allocation or the batch size as function metadata;
- and
- create the database function based on the application logic.
A method for function creation, comprising:
- receiving an indication to create a database function;
- defining one or more database function inputs;
- defining cluster processing information, wherein the cluster processing information includes a performance threshold, an indication to determine resource allocation or batch size of the database function through a static analysis test without executing the database function, an indication to perform a pretest upon execution of the database function to determine resource allocation or batch size, an indication to adjust the resource allocation or the batch size of the database function in the event that performance of the database function is below the performance threshold during execution of the database function, and one or more of the following:
- author specified settings, definition of the static analysis test, and/or a definition of the pretest to be run at execution of the database function;
- defining a deep learning model, comprising:
- specifying a deep learning model object;
- specifying a deep learning model file location and a deep learning model framework, wherein the deep learning model file location specifies where a set of model files associated with the deep learning model is stored, wherein the deep learning model framework comprises a model file format and execution engine type, wherein the deep learning model framework is determined based on a first deep learning model object or a second deep learning model object, wherein computation is performed based on a first deep learning model framework or a second deep learning model framework, and wherein input and output nodes of the deep learning model are determined based on the set of model files and connected to the one or more database function inputs and one or more database function outputs;
- defining the one or more database function outputs;
- and
- creating, using a processor, the database function based at least in part on the one or more database function inputs, the cluster processing information, the deep learning model, and the one or more database function outputs, comprising:
- determining application logic based on the deep learning model;
- determining whether the resource allocation or the batch size is indicated;
- in response to a determination that the resource allocation or the batch size is indicated, storing the resource allocation or the batch size as function metadata;
- and
- creating the database function based on the application logic.
A system for creating a function, comprising:
- an interface configured to:
- receive an indication to create a database function;
- and
- a processor configured to:
- define one or more database function inputs, wherein the one or more database function inputs includes a structured query language (SQL) input;
- define cluster processing information, wherein the cluster processing information includes a performance threshold, an indication to determine resource allocation or batch size of the database function through a static analysis test without executing the database function, an indication to perform a pretest upon execution of the database function to determine resource allocation or batch size, an indication to adjust the resource allocation or the batch size of the database function in the event that performance of the database function is below the performance threshold during execution of the database function, and one or more of the following:
- author specified settings, definition of the static analysis test, and/or a definition of the pretest to be run at execution of the database function;
- define a deep learning model, comprising to:
- specify a deep learning model object;
- specify a deep learning model file location and a deep learning model framework, wherein the deep learning model file location specifies where a set of model files associated with the deep learning model is stored, wherein the deep learning model framework comprises a model file format and execution engine type, wherein the deep learning model framework is determined based on a first deep learning model object or a second deep learning model object, wherein computation is performed based on a first deep learning model framework or a second deep learning model framework, and wherein input and output nodes of the deep learning model are determined based on the set of model files and connected to the one or more database function inputs and one or more database function outputs;
- define the one or more database function outputs, wherein the one or more database function outputs includes one or more of the following:
- a map, vector, and/or an image;
- define a model add-on;
- define a training data set for the model add-on and the deep learning model;
- and
- create a database function based at least in part on the one or more function inputs, the cluster processing information, the deep learning model, the one or more database function outputs, the model add-on, and the training data set, comprising to:
- train, using the training data set, the model add-on when connected to the deep learning model to obtain a second deep learning model;
- determine application logic based on the second deep learning model;
- determine whether the resource allocation or the batch size is indicated;
- in response to a determination that the resource allocation or the batch size is indicated, store the resource allocation or the batch size as function metadata;
- and
- create the database function based on the application logic.
A method for creating a function, comprising:
- receiving an indication to create a database function;
- defining one or more database function inputs, wherein the one or more database function inputs includes a structured query language (SQL) input;
- defining cluster processing information, wherein the cluster processing information includes a performance threshold, an indication to determine resource allocation or batch size of the database function through a static analysis test without executing the database function, an indication to perform a pretest upon execution of the database function to determine resource allocation or batch size, an indication to adjust the resource allocation or the batch size of the database function in the event that performance of the database function is below the performance threshold during execution of the database function, and one or more of the following:
- author specified settings, definition of the static analysis test, and/or a definition of the pretest to be run at execution of the database function;
- defining a deep learning model, comprising:
- specifying a deep learning model object;
- specifying a deep learning model file location and a deep learning model framework, wherein the deep learning model file location specifies where a set of model files associated with the deep learning model is stored, wherein the deep learning model framework comprises a model file format and execution engine type, wherein the deep learning model framework is determined based on a first deep learning model object or a second deep learning model object, wherein computation is performed based on a first deep learning model framework or a second deep learning model framework, and wherein input and output nodes of the deep learning model are determined based on the set of model files and connected to the one or more database function inputs and one or more database function outputs;
- defining the one or more database function outputs, wherein the one or more database function outputs includes one or more of the following:
- a map, vector, and/or an image;
- defining a model add-on;
- defining a training data set for the model add-on and the deep learning model;
- and
- creating, using a processor, a database function based at least in part on the one or more function inputs, the cluster processing information, the deep learning model, the one or more database function outputs, the model add-on, and the training data set, comprising:
- training, using the training data set, the model add-on when connected to the deep learning model to obtain a second deep learning model;
- determining application logic based on the second deep learning model;
- determining whether the resource allocation or the batch size is indicated;
- in response to a determination that the resource allocation or the batch size is indicated, storing the resource allocation or the batch size as function metadata;
- and
- creating the database function based on the application logic.
Dependencies are inferred from the claim text.