# US 2024/0177025 A1 · System and method for managing data processing systems hosting distributed inference models

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0177025 A1
- Document ID: US20240177025A1
- Status: Published application
- Kind: A1
- Application number: 18/060,095
- Filed: 2022-11-30
- Published: 2024-05-30
- Earliest claimed: 2022-11-30
- Assignees: Dell Products L.P.
- Applicants: Dell Products L.P.
- Inventors: OFIR EZRIELEV; JEHUDA SHEMER; TOMER KUSHNIR
- Web page: https://patentgrep.com/patent/US20240177025A1

## CPC

- G06N 5/043: Distributed expert systems; Blackboards
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Methods and systems for managing inference models hosted by data processing systems are disclosed. To manage the inference models, data processing systems that host the inference models may be monitored for risk of compromise. The data processing systems may be monitored by evaluating the processing results generated by the data processing systems. When a data processing system is identified as being compromised, a self-healing process or other remediation process may be initiated to reduce or eliminate the threat to successful inference generation presented by the compromised data processing system.

## Claims

### Claim 1

A method of managing inference models hosted by data processing systems, the method comprising:
obtaining a first processing result from a first data processing system that hosts a portion of a first distributed inference model hosted by a first portion of the data processing systems;
obtaining a second processing result from a second data processing system that hosts a portion of a second distributed inference model hosted by a second portion of the data processing systems;
making a determination, based at least in part on the first processing result and the second processing result, regarding whether the first data processing system is compromised;
in an instance of the determination where the first data processing system is compromised:
initiating self-healing of the first distributed inference model using a third data processing system to remove the first data processing system from membership in the first portion of the data processing systems.

### Claim 2 (depends on 1)

The method of claim 1, wherein making the determination comprises:
making a first comparison between the first processing result and the second processing result;
in an instance of the first comparison where the comparison where the first processing result and the second processing result are different:
obtaining a third processing result from a third data processing system that hosts a portion of a third distributed inference model hosted by a third portion of the data processing systems; and
making a second comparison between the first processing result and the third processing result and a third comparison between the second processing result and the third processing result;
in an instance of the second comparison where the first processing result and the third processing result are different and in an instance of the third comparison where the second processing result and the third processing are not different:
determining that the first data processing system is compromised.

### Claim 3 (depends on 2)

The method of claim 2, wherein the second distributed inference model is redundant for the first distributed inference model, and the third distributed inference model is redundant for the first distributed inference model.

### Claim 4 (depends on 3)

The method of claim 3, wherein the first processing result, the second processing result, and the third processing result are obtained using a same input.

### Claim 5 (depends on 4)

The method of claim 4, further comprising:
making a second determination regarding whether the self-healing has secured the first distributed inference model;
in an instance of the second determination where the first distributed inference model is not secured:
disabling at least the first data processing system; and
rebalancing the data processing systems to host a number of instances of an inference model of a type of the first distributed inference model as specified by a deployment plan for the inference models.

### Claim 6 (depends on 5)

The method of claim 5, wherein the first processing result comprises a partial processing result used by the first distributed inference model to obtain an output.

### Claim 7 (depends on 5)

The method of claim 5, wherein the first processing result comprises an output of the first distributed inference model.

### Claim 8 (depends on 7)

The method of claim 7, wherein the first processing result is a first hash of the output of the first distributed inference model and the second processing result is a second hash of an output of the second distributed inference model.

### Claim 9

A non-transitory machine-readable medium having instructions stored therein, which when executed by a processor, cause the processor to perform operations for managing inference models hosted by data processing systems, the operations comprising:
obtaining a first processing result from a first data processing system that hosts a portion of a first distributed inference model hosted by a first portion of the data processing systems;
obtaining a second processing result from a second data processing system that hosts a portion of a second distributed inference model hosted by a second portion of the data processing systems;
making a determination, based at least in part on the first processing result and the second processing result, regarding whether the first data processing system is compromised;
in an instance of the determination where the first data processing system is compromised:
initiating self-healing of the first distributed inference model using a third data processing system to remove the first data processing system from membership in the first portion of the data processing systems.

### Claim 10 (depends on 9)

The non-transitory machine-readable medium of claim 9, wherein making the determination comprises:
making a first comparison between the first processing result and the second processing result;
in an instance of the first comparison where the comparison where the first processing result and the second processing result are different:
obtaining a third processing result from a third data processing system that hosts a portion of a third distributed inference model hosted by a third portion of the data processing systems; and
making a second comparison between the first processing result and the third processing result and a third comparison between the second processing result and the third processing result;
in an instance of the second comparison where the first processing result and the third processing result are different and in an instance of the third comparison where the second processing result and the third processing are not different:
determining that the first data processing system is compromised.

### Claim 11 (depends on 10)

The non-transitory machine-readable medium of claim 10, wherein the second distributed inference model is redundant for the first distributed inference model, and the third distributed inference model is redundant for the first distributed inference model.

### Claim 12 (depends on 11)

The non-transitory machine-readable medium of claim 11, wherein the first processing result, the second processing result, and the third processing result are obtained using a same input.

### Claim 13 (depends on 12)

The non-transitory machine-readable medium of claim 12, wherein the operations further comprise:
making a second determination regarding whether the self-healing has secured the first data processing system;
in an instance of the second determination where the first distributed inference model is not secured:
disabling at least the first data processing system; and
rebalancing the data processing systems to host a number of instances of an inference model of a type of the first distributed inference model as specified by a deployment plan for the inference models.

### Claim 14 (depends on 13)

The non-transitory machine-readable medium of claim 13, wherein the first processing result comprises a partial processing result used by the first distributed inference model to obtain an output.

### Claim 15 (depends on 13)

The non-transitory machine-readable medium of claim 13, wherein the first processing result comprises an output of the first distributed inference model.

### Claim 16 (depends on 15)

The non-transitory machine-readable medium of claim 15, wherein the first processing result is a first hash of the output of the first distributed inference model and the second processing result is a second hash of an output of the second distributed inference model.

### Claim 17

A data processing system, comprising:
a processor; and
a memory coupled to the processor to store instructions, which when executed by the processor, cause the processor to perform operations for managing a distribution of inference models hosted by data processing systems, the operations comprising:
obtaining a first processing result from a first data processing system that hosts a portion of a first distributed inference model hosted by a first portion of the data processing systems;
obtaining a second processing result from a second data processing system that hosts a portion of a second distributed inference model hosted by a second portion of the data processing systems;
making a determination, based at least in part on the first processing result and the second processing result, regarding whether the first data processing system is compromised;
in an instance of the determination where the first data processing system is compromised:
initiating self-healing of the first distributed inference model using a third data processing system to remove the first data processing system from membership in the first portion of the data processing systems.

### Claim 18 (depends on 9)

The data processing system of claim 9, wherein making the determination comprises:
making a first comparison between the first processing result and the second processing result;
in an instance of the first comparison where the comparison where the first processing result and the second processing result are different:
obtaining a third processing result from a third data processing system that hosts a portion of a third distributed inference model hosted by a third portion of the data processing systems; and
making a second comparison between the first processing result and the third processing result and a third comparison between the second processing result and the third processing result;
in an instance of the second comparison where the first processing result and the third processing result are different and in an instance of the third comparison where the second processing result and the third processing are not different:
determining that the first data processing system is compromised.

### Claim 19 (depends on 10)

The data processing system of claim 10, wherein the second distributed inference model is redundant for the first distributed inference model, and the third distributed inference model is redundant for the first distributed inference model.

### Claim 20 (depends on 11)

The data processing system of claim 11, wherein the first processing result, the second processing result, and the third processing result are obtained using a same input.
