Systems and methods for state map based model instance load balancing
AssigneeProtiviti Inc.
InventorsBenjamin Richard Arroyo Puzon, II, Changbin Vincent Shin
Life of this patent
Estimating the term…
Loading patent…
AssigneeProtiviti Inc.
InventorsBenjamin Richard Arroyo Puzon, II, Changbin Vincent Shin
Estimating the term…
Disclosed herein are a system, a method and a device for providing a state map based model instance load balancing. A server can receive a request from a device in a region to access an instance of an AI model of a plurality of AI models deployed across regions. The server can maintain an AI model map of AI models based at least on the type of AI model. The server can identify, based at least on the request, the region of the request and the type of AI model requested. The server can determine, using the AI model map, the instance of the type of AI model deployed in the region from the plurality of AI models deployed in the region. The server can provide, based at least on the determination, a response to the request providing access to the instance of the type of AI model.
A system comprising:
A method comprising:
A non-transitory computer readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to:
Dependencies are inferred from the claim text.