# US 2026/0064491 A1 · Systems and methods for state map based model instance load balancing

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0064491 A1
- Document ID: US20260064491A1
- Status: Published application
- Kind: A1
- Application number: 19/215,471
- Filed: 2025-05-22
- Published: 2026-03-05
- Earliest claimed: 2024-08-27
- Assignees: Protiviti Inc.
- Applicants: Protiviti Inc.
- Inventors: Benjamin Richard Arroyo Puzon, II; Changbin Vincent Shin
- Web page: https://patentgrep.com/patent/US20260064491A1

## CPC

- G06F 9/5027: {the resource being a machine, e.g. CPUs, Servers, Terminals}
- G06F 9/5083: {Techniques for rebalancing the load in a distributed system}
- G06F 9/5038: {considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration (scheduling strategies G06F9/4881 and subgroups)}
- G06F 9/5072: {Grid computing}
- G06F 9/5088: {involving task migration}
- G06F 2209/502: Proximity
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Disclosed herein are a system, a method and a device for providing a state map based model instance load balancing. A server can receive a request from a device in a region to access an instance of an AI model of a plurality of AI models deployed across regions. The server can maintain an AI model map of AI models based at least on the type of AI model. The server can identify, based at least on the request, the region of the request and the type of AI model requested. The server can determine, using the AI model map, the instance of the type of AI model deployed in the region from the plurality of AI models deployed in the region. The server can provide, based at least on the determination, a response to the request providing access to the instance of the type of AI model.

## Claims

### Claim 1

A system comprising:
a server comprising one or more processors to:
receive a request from a device in a region to access an instance of a type of artificial intelligence (AI) model from a plurality of AI models deployed across a plurality of regions, the server maintaining an AI model map of each instance of an AI model of the plurality of AI models in each region of the plurality of regions based at least on the type of AI model;
identify, based at least on the request, the region of the request and the type of AI model requested;
determine, using the AI model map, the instance of the type of AI model deployed in the region from the plurality of AI models deployed in the region;
provide, based at least on the determination, a response to the request providing access to the instance of the type of AI model.

### Claim 2 (depends on 1)

The system of claim 1, further comprising the one or more processors to:
determine whether the request meets one of a rate of calls for the region and a threshold for number of calls for the region per time period; and
provide access to the instance of the type of AI model deployed in the region based on the determining that the request meets the one of the rate of calls for the region and the threshold for the number of calls for the region per time.

### Claim 3 (depends on 1)

The system of claim 1, further comprising the one or more processors to:
determine whether the request meets one of a rate of calls for the region and a threshold for number of calls for the region per time period; and
determine to provide access to a second instance of the type of AI model in a second region of the plurality of regions responsive to determining that the request does not meet the one of the rate of the calls for the region and the threshold for the number of calls for the region per time.

### Claim 4 (depends on 1)

The system of claim 1, further comprising the one or more processors to:
identify, based on the request, one or more specifications for one or more AI models of the plurality of AI models; and
identify, based on the one or more specifications, the type of AI model requested.

### Claim 5 (depends on 4)

The system of claim 4, further comprising the one or more processors to:
identify, using the AI model map, one or more regions of the plurality of regions that provide the instance of the type of AI model; and
select, based on the region of the request, from the one or more regions, a region of the instance of the type of AI model to generate the response.

### Claim 6 (depends on 5)

The system of claim 5, wherein the region of the instance of the type of AI model is selected based at least on a proximity between the region of the request and the region of the instance of the type of AI model.

### Claim 7 (depends on 1)

The system of claim 1, further comprising the one or more processors to:
detect a geolocation from which the request is originated; and
identify the region of the request based on the geolocation.

### Claim 8 (depends on 1)

The system of claim 1, further comprising the one or more processors to:
determine a match between a region of the instance of the type of AI model and the region of the request; and
determine, using the AI map, to provide access to the instance of the type of AI model based on the match.

### Claim 9 (depends on 1)

The system of claim 1, further comprising the one or more processors to validate, using one or more security control policies, the request.

### Claim 10 (depends on 1)

The system of claim 1, further comprising the one or more processors to:
receive information on status of a plurality of instances of a plurality of AI models, the plurality of instances comprising the instance; and
update, responsive to the information, the AI model map based on the status of the instances of the AI models in the plurality of regions.

### Claim 11 (depends on 1)

The system of claim 1, further comprising the one or more processors to prioritize the instance of the type of AI model based on a proximity of the region of the request to a region in which the instance of the type of AI model is provided.

### Claim 12 (depends on 1)

The system of claim 1, further comprising the one or more processors to:
monitor performance metrics of the plurality of AI models;
adjust the AI model map according to the performance metrics; and
determine, using the AI models map, the instance of AI model based on the performance metrics.

### Claim 13 (depends on 1)

The system of claim 1, further comprising the one or more processors to:
determine a number of instances of the type of AI model provided in the plurality of regions;
determine a number of requests for the number of instances of the type of AI model; and
scale the number of instances of the type of AI model based on the number of requests.

### Claim 14

A method comprising:
receiving, by one or more servers, a request from a device in a region to access an instance of a type of artificial intelligence (AI) model from a plurality of AI models deployed across a plurality of regions, the one or more servers maintaining an AI model map of each instance of an AI model of the plurality of AI models in each region of the plurality of regions based at least on the type of AI model;
identifying, by the one or more servers based at least on the request, the region of the request and the type of AI model requested;
determining, by the one or more servers using the AI model map, the instance of the type of AI model deployed in the region from the plurality of AI models deployed in the region;
providing, by the one or more servers based at least on the determination, a response to the request providing access to the instance of the type of AI model.

### Claim 15 (depends on 14)

The method of claim 14, further comprising:
determining, by the one or more servers, whether the request meets one of a rate of calls for the region and a threshold for number of calls for the region per time period; and
providing, by the one or more servers, access to the instance of the type of AI model deployed in the region based on the determining that the request meets the one of the rate of calls for the region and the threshold for the number of calls for the region per time.

### Claim 16 (depends on 14)

The method of claim 14, further comprising:
determining, by the one or more servers, whether the request meets one of a rate of calls for the region and a threshold for number of calls for the region per time period; and
determining, by the one or more servers, to provide access to a second instance of the type of AI model in a second region of the plurality of regions responsive to determining that the request does not meet the one of the rate of the calls for the region and the threshold for the number of calls for the region per time.

### Claim 17 (depends on 14)

The method of claim 14, further comprising:
identifying, by the one or more servers, based on the request, one or more specifications for one or more AI models of the plurality of AI models; and
identifying, by the one or more servers, based on the one or more specifications, the type of AI model requested.

### Claim 18 (depends on 17)

The method of claim 17, further comprising:
identifying, by the one or more servers, using the AI model map, one or more regions of the plurality of regions that provide the instance of the type of AI model; and
selecting, by the one or more servers based on the region of the request, from the one or more regions, a region of the instance of the type of AI model to generate the response.

### Claim 19 (depends on 18)

The method of claim 18, wherein the region of the instance of the type of AI model is selected based at least on a proximity between the region of the request and the region of the instance of the type of AI model.

### Claim 20

A non-transitory computer readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to:
receive a request from a device in a region to access an instance of a type of artificial intelligence (AI) model from a plurality of AI models deployed across a plurality of regions, the one or more processors accessing an AI model map of each instance of an AI model of the plurality of AI models in each region of the plurality of regions based at least on the type of AI model;
identify, based at least on the request, the region of the request and the type of AI model requested;
determine, using the AI model map, the instance of the type of AI model deployed in the region from the plurality of AI models deployed in the region;
provide, based at least on the determination, a response to the request providing access to the instance of the type of AI model.
