Dynamic load balancer and health monitor for artificial intelligence model deployments
AssigneeBold Limited
InventorsUmar Afzal HAFIZ, Vandana SINGH, Baroon ANAND
Life of this patent
Estimating the term…
Loading patent…
AssigneeBold Limited
InventorsUmar Afzal HAFIZ, Vandana SINGH, Baroon ANAND
Estimating the term…
Certain aspects of the disclosure provide a computer-implemented method for monitoring the health of artificial intelligence (AI) model deployments and provide load balancing of user requests to the AI models. The method periodically sends health-status checks to identify available AI models and unavailable AI models. The method sends request obtained from users to the available AI models that corresponds to an AI model types identified in the requests and within rate limits associated with the available AI models. The method sends answers generated by the available AI models to the users.
A computer-implemented method, comprising:
A processing system, comprising:
An apparatus comprising:
Dependencies are inferred from the claim text.