# US 2023/0266996 A1 · Distributed ai platform mesh for machine learning workflows

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0266996 A1
- Document ID: US20230266996A1
- Status: Published application
- Kind: A1
- Application number: 17/677,588
- Filed: 2022-02-22
- Published: 2023-08-24
- Earliest claimed: 2022-02-22
- Assignees: Sap SE
- Applicants: Sap SE
- Inventors: Srinivasa Byaiah Ramachandra Reddy
- Web page: https://patentgrep.com/patent/US20230266996A1

## CPC

- G06F 9/4881: {Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues}
- G06F 9/5038: {considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration (scheduling strategies G06F9/4881 and subgroups)}
- G06F 9/5072: {Grid computing}
- G06F 9/546: {Message passing systems or structures, e.g. queues}
- G06N 20/00: Machine learning
- G06F 2209/548: Queue
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

An AI platform mesh includes a machine learning control plane hosted in a container orchestration cluster. The machine learning control plane includes a worker plane manager and a set of machine learning services. One or more remote runtime environments configured to execute machine learning workloads can be registered with the machine learning control plane via the worker plane manager. The remote runtime environments can communicate with the set of machine learning services via the worker plane manager.

## Claims

### Claim 1

A system comprising:
a machine learning control plane deployed in a first computing environment, the machine learning control plane comprising a worker plane manager and a set of machine learning services, the worker plane manager comprising a service to register a remote runtime environment and a service to communicate with the remote runtime environment once registered; and
a machine learning worker plane deployed in a second computing environment that is remote to the first computing environment, wherein the machine learning worker plane is a remote runtime environment configured to execute machine learning workloads, and wherein the machine learning worker plane is registered with the machine learning control plane via the worker plane manager.

### Claim 2 (depends on 1)

The system of claim 1, wherein the first computing environment comprises a container orchestration cluster, and wherein the machine learning control plane is hosted in the container orchestration cluster.

### Claim 3 (depends on 2)

The system of claim 2, wherein the machine learning control plane comprises a set of containerized microservices running in replica pods in the container orchestration cluster.

### Claim 4 (depends on 2)

The system of claim 2, wherein the machine learning worker plane comprises a machine learning service broker configured to create and manage machine learning workloads on the machine learning worker plane.

### Claim 5 (depends on 4)

The system of claim 4, wherein the machine learning service broker integrates with one or more container orchestration APIs to manage the machine learning workloads on the machine learning worker plane.

### Claim 6 (depends on 5)

The system of claim 5, wherein the machine learning service broker integrates with one or more hyperscaler APIs to manage the machine learning workloads on the machine learning worker plane.

### Claim 7 (depends on 4)

The system of claim 4, wherein the machine learning workloads comprise an inference service.

### Claim 8 (depends on 4)

The system of claim 4, wherein the machine learning service broker is communicatively coupled to the set of machine learning services through the worker plane manager.

### Claim 9 (depends on 8)

The system of claim 8, wherein the machine learning service broker is connected to the worker plane manager via a synchronization service or a message queue service.

### Claim 10 (depends on 1)

The system of claim 1, wherein the set of machine learning services comprises services to preprocess machine learning data, train machine learning models, deploy trained models, manage machine learning artifacts, and manage machine learning workflows.

### Claim 11 (depends on 1)

The system of claim 1, wherein the second computing environment is an edge computing environment, a public cloud computing environment, or a private cloud computing environment.

### Claim 12 (depends on 1)

The system of claim 1, wherein the machine learning control plane further comprises a set of common services comprising a logging service, a machine learning tracking service, a metrics service, a tracing service, and an audit service.

### Claim 13 (depends on 12)

The system of claim 12, wherein the machine learning worker plane further comprises a common service agent configured to broadcast system data from the machine learning worker plane to the set of common services, and wherein the common service agent is communicatively coupled to the set of common services via the worker plane manager.

### Claim 14

A method comprising:
running a machine learning control plane in a container orchestration cluster, wherein the machine learning control plane comprises a worker plane manager and a set of machine learning services;
receiving a request to register a machine learning worker plane with the machine learning control plane, wherein the machine learning worker plane is a remote runtime environment configured to execute machine learning workloads;
registering the machine learning worker plane in the worker plane manager; and
configuring a communication connection between the worker plane manager and a machine learning service broker within the machine learning worker plane.

### Claim 15 (depends on 14)

The method of claim 14, further comprising creating a machine learning scenario within a machine learning workflow on the machine learning control plane, the machine learning scenario comprising one or more machine learning services to apply to one or more machine learning artifacts.

### Claim 16 (depends on 15)

The method of claim 15, further comprising:
executing the machine learning scenario on the machine learning control plane; and
communicating information about the execution of the machine learning scenario from the machine learning control plane to the machine learning worker plane via the communication connection.

### Claim 17 (depends on 15)

The method of claim 15, wherein the one or more machine learning artifacts comprise a machine learning model, and wherein the one or more machine learning services comprise training of the machine learning model.

### Claim 18 (depends on 17)

The method of claim 17, further comprising creating a machine learning workload on the machine learning worker plane for an inference service based on the machine learning model.

### Claim 19 (depends on 17)

The method of claim 17, further comprising delegating the training of the machine learning model to the machine learning worker plane.

### Claim 20

One or more non-transitory computer-readable storage media storing computer-executable instructions for causing a computer system to perform operations comprising:
receiving a first request at a machine learning control plane deployed in a container orchestration cluster to register a remote runtime environment configured to execute machine learning workloads;
registering the remote runtime environment at the machine learning control plane;
receiving a second request at the machine learning control plane, the second request comprising a machine learning scenario specifying one or more machine learning services of the machine learning control plane;
executing the machine learning scenario on the machine learning control plane;
communicating information about execution of the machine learning service from the machine learning control plane to the remote runtime environment; and
communicating information about machine learning workloads on the remote runtime environment to the machine learning control plane.
