Machine learning pipelines
AssigneeAmazon Technologies, Inc.
InventorsOwen Thomas, Arun Babu Nagarajan, Kenneth O Henderson, Weixun Wang, Urvashi Chowdhary
Life of this patent
Estimating the term…
Abstract
Techniques described herein may be implemented in the context of a computing resource service provider. A machine learning (ML) service provides an interface to clients which can be used to create, read, update, and delete ML pipelines. ML pipelines can be converted to a human-readable format, which a server persists. Clients of a machine learning service can start, stop, and resume executions a ML pipeline based on the human-readable format of the ML pipeline.
Claims20
A computer-implemented method, comprising:
- obtaining a programmatic structure of a machine learning (ML) pipeline according to a first programming syntax, the programmatic structure of the ML pipeline comprising data defining a workflow of operations:
- obtain data to be used to train an ML model;
- train the ML model using the data;
- and
- deploy the ML model to perform inferencing;
- processing, according to a second programming syntax, the programmatic structure of the ML pipeline to generate a data interchange format of the ML pipeline, the data interchange format of the ML pipeline comprising a graph representation of the programmatic structure of the ML pipeline, the graph representation comprising at least one node representing at least one of the operations of the workflow, processing the programmatic structure of the ML pipeline including at least using a compiler that converts programing language code of the programmatic structure of the ML pipeline to generate the data interchange format of the ML pipeline;
- and
- causing one or more services to execute at least one of the operations of the workflow based on the data interchange format of the ML pipeline, execution of at least one of the operations triggering obtaining the data to be used to train the ML model, training the ML model using the data, or deploying the ML model to perform the inferencing.
A system, comprising:
- one or more processors;
- and
- memory that stores computer-executable instructions that are executable by the one or more processors to cause the system to:
- receive a first representation of a machine learning (ML) pipeline, the ML pipeline comprising at least one operation to be executed;
- convert the first representation of the ML pipeline to generate a second representation of the ML pipeline, converting the first representation of the ML pipeline including at least using a compiler that converts programing language code of the first representation of the ML pipeline to generate the second representation of the ML pipeline based on a data interchange format;
- generate a visual representation based on the second representation of the ML pipeline, the visual representation comprising at least one node, the at least one node corresponding to the at least one operation to be executed;
- and
- cause the at least one operation to be executed according to the second representation of the ML pipeline, the at least one operation to obtain data to be used to train an ML model, train the ML model, or deploy the ML model to perform inferencing.
A non-transitory computer-readable storage medium storing thereon executable instructions that, as a result of being executed by one or more processors of a computer system, cause the computer system to:
- obtain a programmatically defined workflow structure associated with a machine learning (ML) process, the programmatically defined workflow structure comprising at least one ML operation to be executed;
- process the programmatically defined workflow structure to generate, based on a data interchange format, a textualized workflow structure corresponding to the programmatically defined workflow structure, processing the programmatically defined workflow structure to generate the textualized workflow structure including using a code compiler that converts programming language code of the programmatically defined workflow structure to generate the textualized workflow structure;
- and
- cause the at least one ML operation to be executed according to the textualized workflow structure, the at least one ML operation to obtain data to be used to train an ML model, train the ML model, or deploy the ML model to perform inferencing.
Dependencies are inferred from the claim text.