# US 12,001,823 B2 · Systems and methods for building and deploying machine learning applications > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,001,823 B2 - Document ID: US12001823B2 - Status: Granted - Kind: B2 - Application number: 18/317,179 - Filed: 2023-05-15 - Published: 2024-03-14 - Granted: 2024-06-04 - Earliest claimed: 2020-12-24 - Assignees: Sway AI Inc. - Applicants: Sway AI Inc. - Inventors: Jitender Arora; Amir H. Atai; Michael Piecuch; Hassan Ahmed - Web page: https://patentgrep.com/patent/US12001823B2 ## CPC - G06F 8/60: Software deployment - G06N 20/20: Ensemble learning - G06F 8/34: Graphical or visual programming - G06F 9/4881: {Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues} - G06F 9/5038: {considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration (scheduling strategies G06F9/4881 and subgroups)} - G06N 5/01: Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound - G06N 20/00: Machine learning - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract According to some aspects, techniques are provided for building and deploying a machine learning application that do not require a user to have expert knowledge of machine learning or programming. These techniques may be executed by a system that provides a graphical user interface which allows a user to visually define a workflow for a machine learning application, without requiring the user to be an expert in machine learning. The system may automatically represent the workflow as a specification that may be used to build and deploy a machine learning application. The system may automatically execute the workflow in a series of stages while managing data flow and execution context between the stages. Such an execution process may provide flexibility in execution so that a user can build a complex machine learning application without it being necessary for the user to have detailed knowledge of how execution is managed. ## Claims ### Claim 1 A method of operating a data processing system configured to deploy machine learning applications according to respective machine learning workflow specifications, each machine learning workflow specification defining a plurality of components of a corresponding machine learning workflow and defining one or more connections between the plurality of components, each of the plurality of components defining an operation to be performed on data and having an associated context, and each of the one or more connections defining a flow of data between components when the corresponding machine learning workflow is executed, the method comprising, by one or more processors: identifying a first group of one or more components of a first machine learning workflow as being associated with a first context, based on a first machine learning workflow specification that represents the first machine learning workflow; executing, or causing execution of, the one or more components of the first group in an order according to one or more connections between the one or more components of the first group as defined by the first machine learning workflow specification; identifying a second group of one or more components of the first machine learning workflow as being associated with a second context, based on the first machine learning workflow specification; and executing, or causing execution of, the one or more components of the second group in an order according to one or more connections among the one or more components of the second group as defined by the first machine learning workflow specification. ### Claim 2 (depends on 1) The method of claim 1, wherein: each of the plurality of components of the first machine learning workflow comprises an input data port and an output data port; and each connection of the one or more connections between the plurality of components defines a flow of data from an output data port of one component of the plurality of components of the first machine learning workflow to an input data port of another component of the plurality of components of the first machine learning workflow. ### Claim 3 (depends on 2) The method of claim 2, further comprising: obtaining the first machine learning workflow specification, via a graphical user interface, by: receiving first user input identifying the one or more components of the first machine learning workflow; and receiving second user input defining the one or more connections among the one or more components; and generating the first machine learning workflow specification based on the first user input and the second user input. ### Claim 4 (depends on 3) The method of claim 3, wherein obtaining the first machine learning workflow specification further comprises: validating each of the one or more connections by determining that a data type of the output data port of the first component of the plurality of components of the first machine learning application matches a data type of the input data port of the second component of the plurality of components of the first machine learning workflow. ### Claim 5 (depends on 1) The method of claim 1, further comprising: determining that execution of the one or more components of the first group has completed before executing the one or more components of the second group, wherein: the one or more components in the first group, when executed, perform a first operation selected from a list of operations comprising: data ingestion, data preparation, data labeling, machine learning model training, machine learning model evaluation, machine learning inference, business logic, and machine learning model deployment; and the one or more components in the second group, when executed, perform a second operation selected from the list of operations, wherein the second operation is subsequent to the first operation in the list of operations. ### Claim 6 (depends on 1) The method of claim 1, wherein: executing the one or more components of the first group is performed by a first processor having a first speed; and executing the one or more components of the second group is performed by a second processor having a second speed different from the first speed. ### Claim 7 (depends on 1) The method of claim 1, wherein: executing the one or more components of the first group comprises in part storing output data in a shared storage; and executing the one or more components of the second group comprises in part accessing the output data stored in the shared storage. ### Claim 8 (depends on 1) The method of claim 1, wherein executing the one or more components of the first group comprises: dynamically generating an executable program configured to execute the one or more components of the first group; and causing the executable program to execute the one or more components of the first group. ### Claim 9 A system comprising: one or more processors configured to execute programming instructions to perform operations to deploy machine learning applications according to respective machine learning workflow specifications, each machine learning workflow specification defining a plurality of components of a corresponding machine learning workflow and defining one or more connections between the plurality of components, each of the plurality of components defining an operation to be performed on data and having an associated context, and each of the one or more connections defining a flow of data between components when the corresponding machine learning workflow is executed, the operations comprising: identifying a first group of one or more components of a first machine learning workflow as being associated with a first context, based on a first machine learning workflow specification that represents the first machine learning workflow; executing, or causing execution of, the one or more components of the first group in an order according to one or more connections between the one or more components of the first group as defined by the first machine learning workflow specification; identifying a second group of one or more components of the first machine learning workflow as being associated with a second context, based on the first machine learning workflow specification; and executing, or causing execution of, the one or more components of the second group in an order according to one or more connections among the one or more components of the second group as defined by the first machine learning workflow specification. ### Claim 10 (depends on 9) The system of claim 9, wherein: each of the plurality of components of the first machine learning workflow comprises an input data port and an output data port; and each connection of the one or more connections between the plurality of components defines a flow of data from an output data port of one component of the plurality of components of the first machine learning workflow to an input data port of another component of the plurality of components of the first machine learning workflow. ### Claim 11 (depends on 10) The system of claim 10, further comprising: a graphical user interface; wherein the operations further comprise: configuring the graphical user interface to obtain the first machine learning workflow specification by: receiving first user input identifying the one or more components of the first machine learning workflow; and receiving second user input defining the one or more connections among the one or more components; and generating the first machine learning workflow specification based on the first user input and the second user input. ### Claim 12 (depends on 11) The system of claim 11, wherein graphical user interface is further configured to: validate each of the one or more connections by determining that a data type of the output data port of the first component of the plurality of components of the first machine learning application matches a data type of the input data port of the second component of the plurality of components of the first machine learning workflow. ### Claim 13 (depends on 9) The system of claim 9, wherein the operations further comprise: determining that execution of the one or more components of the first group has completed before executing the one or more components of the second group, wherein: the one or more components in the first group, when executed, perform a first operation selected from a list of operations comprising: data ingestion, data preparation, data labeling, machine learning model training, machine learning model evaluation, machine learning inference, business logic, and machine learning model deployment; and the one or more components in the second group, when executed, perform a second operation selected from the list of operations, wherein the second operation is subsequent to the first operation in the list of operations. ### Claim 14 (depends on 9) The system of claim 9, wherein: executing the one or more components of the first group is performed by a first processor having a first speed; and executing the one or more components of the second group is performed by a second processor having a second speed different from the first speed. ### Claim 15 (depends on 9) The system of claim 9, wherein: executing the one or more components of the first group comprises in part storing output data in a shared storage; and executing the one or more components of the second group comprises in part accessing the output data stored in the shared storage. ### Claim 16 (depends on 9) The system of claim 9, wherein executing the one or more components of the first group comprises: dynamically generating an executable program configured to execute the one or more components of the first group; and causing the executable program to execute the one or more components of the first group. ### Claim 17 At least one non-transitory computer-readable medium comprising instructions that, when executed by one or more processors on a computing device, are operable to cause the one or more processors to perform operations to deploy machine learning applications according to respective machine learning workflow specifications, each machine learning workflow specification defining a plurality of components of a corresponding machine learning workflow and defining one or more connections between the plurality of components, each of the plurality of components defining an operation to be performed on data and having an associated context, and each of the one or more connections defining a flow of data between components when the corresponding machine learning workflow is executed, the operations comprising: identifying a first group of one or more components of a first machine learning workflow as being associated with a first context, based on a first machine learning workflow specification that represents the first machine learning workflow; executing, or causing execution of, the one or more components of the first group in an order according to one or more connections between the one or more components of the first group as defined by the first machine learning workflow specification; identifying a second group of one or more components of the first machine learning workflow as being associated with a second context, based on the first machine learning workflow specification; and executing, or causing execution of, the one or more components of the second group in an order according to one or more connections among the one or more components of the second group as defined by the first machine learning workflow specification. ### Claim 18 (depends on 17) The at least one non-transitory computer-readable medium of claim 17, wherein: each of the plurality of components of the first machine learning workflow comprises an input data port and an output data port; and each connection of the one or more connections between the plurality of components defines a flow of data from an output data port of one component of the plurality of components of the first machine learning workflow to an input data port of another component of the plurality of components of the first machine learning workflow. ### Claim 19 (depends on 18) The at least one non-transitory computer-readable medium of claim 18, wherein the operations further comprise: obtaining the first machine learning workflow specification, via a graphical user interface, by: receiving first user input identifying the one or more components of the first machine learning workflow; and receiving second user input defining the one or more connections among the one or more components; and generating the first machine learning workflow specification based on the first user input and the second user input. ### Claim 20 (depends on 19) The at least one non-transitory computer-readable medium of claim 19, wherein obtaining the first machine learning workflow specification further comprises: validating each of the one or more connections by determining that a data type of the output data port of the first component of the plurality of components of the first machine learning application matches a data type of the input data port of the second component of the plurality of components of the first machine learning workflow. ### Claim 21 (depends on 17) The at least one non-transitory computer-readable medium of claim 17, wherein the operations further comprise: determining that execution of the one or more components of the first group has completed before executing the one or more components of the second group, wherein: the one or more components in the first group, when executed, perform a first operation selected from a list of operations comprising: data ingestion, data preparation, data labeling, machine learning model training, machine learning model evaluation, machine learning inference, business logic, and machine learning model deployment; and the one or more components in the second group, when executed, perform a second operation selected from the list of operations, wherein the second operation is subsequent to the first operation in the list of operations. ### Claim 22 (depends on 17) The at least one non-transitory computer-readable medium of claim 17, wherein: executing the one or more components of the first group is performed by a first processor having a first speed; and executing the one or more components of the second group is performed by a second processor having a second speed different from the first speed. ### Claim 23 (depends on 17) The at least one non-transitory computer-readable medium of claim 17, wherein: executing the one or more components of the first group comprises in part storing output data in a shared storage; and executing the one or more components of the second group comprises in part accessing the output data stored in the shared storage. ### Claim 24 (depends on 17) The at least one non-transitory computer-readable medium of claim 17, wherein executing the one or more components of the first group comprises: dynamically generating an executable program configured to execute the one or more components of the first group; and causing the executable program to execute the one or more components of the first group.