# US 2024/0177027 A1 · System and method for managing inference model performance through proactive communication system analysis

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0177027 A1
- Document ID: US20240177027A1
- Status: Published application
- Kind: A1
- Application number: 18/060,122
- Filed: 2022-11-30
- Published: 2024-05-30
- Earliest claimed: 2022-11-30
- Assignees: Dell Products L.P.
- Applicants: Dell Products L.P.
- Inventors: OFIR EZRIELEV; JEHUDA SHEMER; TOMER KUSHNIR
- Web page: https://patentgrep.com/patent/US20240177027A1

## CPC

- G06N 5/043: Distributed expert systems; Blackboards
- G06N 20/00: Machine learning
- H04L 41/145: {involving simulating, designing, planning or modelling of a network}
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

Methods and systems for managing execution of inference models hosted by data processing systems are disclosed. To manage execution of inference models hosted by data processing systems, a system may include an inference model manager and any number of data processing systems. The inference model manager may communication system data for the communication system linking the data processing systems. The inference model manager may use the communication system data to determine whether the communication system meets inference generation requirements of the downstream consumer. If the communication system does not meet inference generation requirements of the downstream consumer, the inference model manager may obtain an inference generation plan to return to compliance with the inference generation requirements of the downstream consumer.

## Claims

### Claim 1

A method of managing execution of an inference model hosted by data processing systems, the method comprising:
obtaining communication system information for a communication system connecting the data processing systems;
making a determination regarding whether the communication system information meets inference generation requirements of a downstream consumer;
in an instance of the determination, in which the communication system information does not meet the inference generation requirements of the downstream consumer:
obtaining an inference generation path for the inference model based on the inference generation requirements of the downstream consumer and the communication system information; and
modifying a deployment of the inference model to the data processing systems based on the inference generation path.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
prior to obtaining the communication system information:
obtaining the inference model;
obtaining characteristics of the inference model and characteristics of the data processing systems;
obtaining portions of the inference model based on the characteristics of the data processing systems and the characteristics of the inference model;
obtaining an execution plan for the inference model based on:
the portions of the inference model,
the characteristics of the data processing systems, and
the inference generation requirements of the downstream consumer; and
distributing the portions of the inference model to the data processing systems based on the execution plan.

### Claim 3 (depends on 2)

The method of claim 2, wherein the communication system information comprises:
a quantity of available communication system bandwidth between each data processing system of the data processing systems; and
a reliability of transmission between each data processing system of the data processing systems.

### Claim 4 (depends on 3)

The method of claim 3, wherein the reliability of transmission is based on:
historical data indicating a likelihood of successful transmission of data between each data processing system of the data processing systems; or
a distance between each data processing system of the data processing systems.

### Claim 5 (depends on 4)

The method of claim 4, wherein the inference generation requirements of the downstream consumer are based on:
an inference generation speed threshold, and
an inference generation reliability threshold.

### Claim 6 (depends on 5)

The method of claim 5, wherein the inference generation speed threshold indicates a minimum quantity of communication bandwidth between each data processing system of the data processing systems to meet the inference generation requirements of the downstream consumer.

### Claim 7 (depends on 6)

The method of claim 6, wherein the inference generation reliability threshold indicates a minimum likelihood of successful transmission of data between each data processing system of the data processing systems to meet the inference generation requirements of the downstream consumer.

### Claim 8 (depends on 7)

The method of claim 7, wherein the inference generation path comprises:
a listing of instances of each of the portions of the inference model usable to generate an inference model result in compliance with the inference generation requirements of the downstream consumer; and
an ordering of the listing of the instances.

### Claim 9 (depends on 8)

The method of claim 8, wherein modifying the deployment of the inference model comprises:
generating an updated execution plan based on the inference generation path; and
distributing the updated execution plan to the data processing systems to implement the updated execution plan.

### Claim 10 (depends on 9)

The method of claim 9, wherein the communication system comprises:
multiple point-to-point wireless connections between the data processing systems, each point-to-point wireless connection of the multiple point-to-point wireless connections having distinct characteristics.

### Claim 11

A non-transitory machine-readable medium having instructions stored therein, which when executed by a processor, cause the processor to perform operations for managing execution of an inference model hosted by data processing systems, the operations comprising:
obtaining communication system information for a communication system connecting the data processing systems;
making a determination regarding whether the communication system information meets inference generation requirements of a downstream consumer;
in an instance of the determination, in which the communication system information does not meet the inference generation requirements of the downstream consumer:
obtaining an inference generation path for the inference model based on the inference generation requirements of the downstream consumer and the communication system information; and
modifying a deployment of the inference model to the data processing systems based on the inference generation path.

### Claim 12 (depends on 11)

The non-transitory machine-readable medium of claim 11, further comprising:
prior to obtaining the communication system information:
obtaining the inference model;
obtaining characteristics of the inference model and characteristics of the data processing systems;
obtaining portions of the inference model based on the characteristics of the data processing systems and the characteristics of the inference model;
obtaining an execution plan for the inference model based on:
the portions of the inference model,
the characteristics of the data processing systems, and
the inference generation requirements of the downstream consumer; and
distributing the portions of the inference model to the data processing systems based on the execution plan.

### Claim 13 (depends on 12)

The non-transitory machine-readable medium of claim 12, wherein the communication system information comprises:
a quantity of available communication system bandwidth between each data processing system of the data processing systems; and
a reliability of transmission between each data processing system of the data processing systems.

### Claim 14 (depends on 13)

The non-transitory machine-readable medium of claim 13, wherein the reliability of transmission is based on:
historical data indicating a likelihood of successful transmission of data between each data processing system of the data processing systems; or
a distance between each data processing system of the data processing systems.

### Claim 15 (depends on 14)

The non-transitory machine-readable medium of claim 14, wherein the inference generation requirements of the downstream consumer are based on:
an inference generation speed threshold, and
an inference generation reliability threshold.

### Claim 16

A data processing system, comprising:
a processor; and
a memory coupled to the processor to store instructions, which when executed by the processor, cause the processor to perform operations for managing execution of an inference model hosted by data processing systems, the operations comprising:
obtaining communication system information for a communication system connecting the data processing systems;
making a determination regarding whether the communication system information meets inference generation requirements of a downstream consumer;
in an instance of the determination, in which the communication system information does not meet the inference generation requirements of the downstream consumer:
obtaining an inference generation path for the inference model based on the inference generation requirements of the downstream consumer and the communication system information; and
modifying a deployment of the inference model to the data processing systems based on the inference generation path.

### Claim 17 (depends on 16)

The data processing system of claim 16, further comprising:
prior to obtaining the communication system information:
obtaining the inference model;
obtaining characteristics of the inference model and characteristics of the data processing systems;
obtaining portions of the inference model based on the characteristics of the data processing systems and the characteristics of the inference model;
obtaining an execution plan for the inference model based on:
the portions of the inference model,
the characteristics of the data processing systems, and
the inference generation requirements of the downstream consumer; and
distributing the portions of the inference model to the data processing systems based on the execution plan.

### Claim 18 (depends on 17)

The data processing system of claim 17, wherein the communication system information comprises:
a quantity of available communication system bandwidth between each data processing system of the data processing systems; and
a reliability of transmission between each data processing system of the data processing systems.

### Claim 19 (depends on 18)

The data processing system of claim 18, wherein the reliability of transmission is based on:
historical data indicating a likelihood of successful transmission of data between each data processing system of the data processing systems; or
a distance between each data processing system of the data processing systems.

### Claim 20 (depends on 19)

The data processing system of claim 19, wherein the inference generation requirements of the downstream consumer are based on:
an inference generation speed threshold, and
an inference generation reliability threshold.
