# US 12,762,483 B2 · Systems and methods for integrated orchestration of machine learning operations > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,762,483 B2 - Document ID: US12762483B2 - Status: Granted - Kind: B2 - Application number: 17/950,871 - Filed: 2022-09-22 - Published: 2023-03-30 - Granted: 2026-10-13 - Earliest claimed: 2021-09-22 - Assignees: RPS Canada Inc. - Applicants: RPS Canada Inc. - Inventors: Joshua Geist; Matthew Goddard - Web page: https://patentgrep.com/patent/US12762483B2 ## CPC - G06N 20/20: Ensemble learning - G06F 9/4881: {Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues} - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Disclosed herein are systems and methods for managing and deploying pre-trained machine-learning (ML) models from disparate third-party systems and a host system. An orchestration system selects effective pre-trained ML models to suit a client user's business operation demands and data analysis requirements. A model orchestration engine accesses and maintains a model orchestration database containing a catalogue of ML models. The catalog indicates, for example, the capabilities, data inputs, and data outputs of each ML model. The orchestration engine builds an execution pipeline of certain ML models according to a client request for an operation or function and the client data submitted from a client device. The orchestration server can format or normalize the client data or output data received from the client device or an earlier model according to a data specification, thereby generating input data for subsequent models in the pipeline. ## Claims ### Claim 1 A computer-implemented method comprising: receiving, by a computer, from a client device a request for an operation using client data; selecting, by the computer, from a catalog of machine learning models stored in an orchestration database, a plurality of machine learning models based upon the operation of the request, the catalog indicating a set of one or more features corresponding to the operation for each machine learning model of the catalog; generating, by the computer, an execution pipeline having a plurality of data records corresponding to the plurality of machine learning models hosted on a plurality of host servers, each data record indicating a data specification corresponding to a machine learning model and a resource locator for a host server hosting the machine learning model; formatting, by the computer, the client data as input data for a first machine learning model of the plurality of machine learning models according to the data specification of the first machine learning model indicated by the execution pipeline; and iteratively executing, by the computer, the plurality of machine learning models of the execution pipeline, comprising formatting output data received from the host server hosting a preceding machine learning model as the input data for a subsequent machine learning model of the plurality of machine learning models, and transmitting the input data to the host server hosting the subsequent machine learning model according to the resource locator indicated by the execution pipeline. ### Claim 2 (depends on 1) The method according to claim 1, wherein the computer iteratively executes the execution pipeline for a plurality of iterations, and wherein the method further comprises, for each iteration of at least one iteration: transmitting, by the computer, the input data to the host server hosting the machine learning model to be executed for the iteration. ### Claim 3 (depends on 2) The method according to claim 2, further comprising: identifying, by the computer, a data-transfer requirement for the host server of the machine learning model, wherein the computer transmits the input data to the host server according to the data-transfer requirement. ### Claim 4 (depends on 1) The method according to claim 1, for each iteration of at least one iteration: receiving, by the computer, the output data resulting from the machine learning model of the iteration from a host server hosting the machine learning model. ### Claim 5 (depends on 1) The method according to claim 1, for each iteration of at least one iteration: generating, by the computer, the input data for the machine learning model of the iteration based upon formatting the output data of the preceding machine learning model of a preceding iteration. ### Claim 6 (depends on 1) The method according to claim 1, further comprising: identifying, by the computer, a data specification for the machine learning model of a host server, wherein the input data is formatted by the computer for the machine learning model according to the data specification. ### Claim 7 (depends on 1) The method according to claim 1, further comprising: identifying, by the computer, a data specification for the machine learning model of a host server; and selecting, by the computer, the machine learning model of the execution pipeline based upon a type of file of the client data received from the client device. ### Claim 8 (depends on 1) The method according to claim 1, further comprising: identifying, by the computer, a data specification for the machine learning model of a host of the machine learning model; and selecting, by the computer, the machine learning model of the execution pipeline based upon a type of the output data of a next machine learning model selected for the execution pipeline. ### Claim 9 (depends on 1) The method according to claim 1, further comprising: determining, by the computer, an order of execution for the plurality of machine learning models of the execution pipeline based upon the request from the client device. ### Claim 10 (depends on 1) The method according to claim 1, wherein at least two machine learning models are executed in parallel for a particular iteration using the input data. ### Claim 11 A system comprising: a computer comprising a processor configured to: receive from a client device a request for an operation using client data; select, from a catalog of machine learning models stored in an orchestration database, a plurality of machine learning models based upon the operation of the request, the catalog indicating a set of one or more features corresponding to the operation for each machine learning model of the catalog; generate an execution pipeline having a plurality of data records corresponding to the plurality of machine learning models hosted on a plurality of host servers, each data record indicating a data specification corresponding to a machine learning model and a resource locator for a host server hosting the machine learning model; format the client data as input data for a first machine learning model of the plurality of machine learning models according to the data specification of the first machine learning model indicated by the execution pipeline; and iteratively execute the plurality of machine learning models of the execution pipeline, comprising formatting output data received from the host server hosting a preceding machine learning model as the input data for a subsequent machine learning model of the plurality of machine learning models, and transmitting the input data to the host server hosting the subsequent machine learning model according to the resource locator indicated by the execution pipeline. ### Claim 12 (depends on 11) The system according to claim 11, wherein the computer iteratively executes the execution pipeline for a plurality of iterations, and wherein the computer is further configured to, for each iteration of at least one iteration: transmit the input data to the host server hosting the machine learning model to be executed for the iteration. ### Claim 13 (depends on 12) The system according to claim 12, wherein the computer is further configured to identify a data-transfer requirement for the host server of the machine learning model, wherein the computer transmits the input data to the host server according to the data-transfer requirement. ### Claim 14 (depends on 11) The system according to claim 11, wherein the computer is further configured to, for each iteration of at least one iteration: receive the output data resulting from the machine learning model of the iteration from a host server hosting the machine learning model. ### Claim 15 (depends on 11) The system according to claim 11, wherein the computer is further configured to, for each iteration of at least one iteration: generate the input data for the machine learning model of the iteration based upon formatting the output data of the preceding machine learning model of a preceding iteration. ### Claim 16 (depends on 11) The system according to claim 11, wherein the computer is further configured to identify a data specification for the machine learning model of a host server, wherein the input data is formatted by the computer for the machine learning model according to the data specification. ### Claim 17 (depends on 11) The system according to claim 11, wherein the computer is further configured to: identify a data specification for the machine learning model of a host server; and select the machine learning model of the execution pipeline based upon a type of file of the client data received from the client device. ### Claim 18 (depends on 11) The system according to claim 11, wherein the computer is further configured to: identify a data specification for the machine learning model of a host of the machine learning model; and select the machine learning model of the execution pipeline based upon a type of the output data of a next machine learning model selected for the execution pipeline. ### Claim 19 (depends on 11) The system according to claim 11, wherein the computer is further configured to determine an order of execution for the plurality of machine learning models of the execution pipeline based upon the request from the client device. ### Claim 20 (depends on 11) The system according to claim 11, wherein at least two machine learning models are executed in parallel for a particular iteration using the input data.