Peer-to-peer load balancing for faas in agentic ai systems
AssigneeNEC Laboratories America, Inc.
InventorsGiuseppe Coviello, Kunal Rao, Mohammad Khojastepour, Srimat Chakradhar
Life of this patent
Estimating the term…
Abstract
Systems and methods for peer-to-peer load balancing. The systems and methods include prompting a model to develop a workflow, the workflow including a set of component operations that can be executed as more than one request on more than one pod and queueing the more than one request for execution on the more than one pod. The systems and methods further include comparing a first pod request queue with a second pod request queue, and in response to the comparison, transferring one of the more than one request from a first pod request queue to a second pod request queue and executing the more than one request.
Claims20
A method, comprising:
- prompting a model to develop a workflow, the workflow including a set of component operations that can be executed as more than one request on more than one pod;
- queueing the more than one request for execution on the more than one pod;
- comparing a first pod request queue with a second pod request queue, and in response to the comparison, transferring one of the more than one request from a first pod request queue to a second pod request queue;
- and
- executing the more than one request.
A system, comprising:
- a processor;
- and
- a memory storing computer-readable instructions that, when executed by the processor, cause the processor to:
- prompt a model to develop a workflow, the workflow includes a set of component operations that can be executed as more than one request on more than one pod;
- queue the more than one request for execution on the more than one pod;
- compare a first pod request queue with a second pod request queue, and in response to the comparison, transferring one of the more than one request from a first pod request queue to a second pod request queue;
- and
- execute the more than one request.
A computer program product comprising a non-transitory computer-readable storage medium containing computer program code, the computer program code when executed by one or more processors causes the one or more processors to perform operations, the computer program code comprising instructions to:
- prompt a model to develop a workflow, the workflow includes a set of component operations that can be executed as more than one request on more than one pod;
- queue the more than one request for execution on the more than one pod;
- compare a first pod request queue with a second pod request queue, and in response to the comparison, transferring one of the more than one request from a first pod request queue to a second pod request queue;
- and
- execute the more than one request.
Dependencies are inferred from the claim text.