Back to results
US 12,754,765 B2Granted
Stateful artificial intelligence (AI) runtime environment for isolated stateful execution of al agent workflows
AssigneeINK Content, Inc.
InventorsAlexander De Ridder, Alaa-eddine Kaddouri
Life of this patent
Estimating the term…
Abstract
The techniques herein provide for an AI agent-centric stateful runtime environment with integrated memory management and dynamic serializing and restoring of non-deterministic agent computational state across asynchronous execution boundaries. The approach herein overcomes the inherent statelessness and context-window limitations of Large Language Models (LLMs) when such models are utilized in agentic workflows.
Claims20
A method implemented in a computing system having a runtime execution environment, the runtime execution environment configured to provide stateful agentic orchestration of a set of Artificial Intelligence (AI) agents by:
- responsive to receipt of a request associated with an AI agent workflow, wherein the request is associated with a configuration, instantiating an isolated stateful execution session;
- parsing the configuration into a set of one or more discrete computational operations for a set of AI agents;
- orchestrating execution of the set of one or more discrete computational operations within the isolated stateful execution session;
- during the execution of the set of or more discrete computational operations, detecting whether an execution boundary associated with a given AI agent has been reached;
- upon determining that the execution boundary associated with the given AI agent has been reached, suspending the isolated stateful execution session;
- serializing to a memory state data associated with the given AI agent;
- thereafter, determining whether to resume the isolated stateful execution session with the given AI agent;
- upon a determination to resume the isolated stateful execution session, re-hydrating the given AI agent, at least in part using the state data that was previously serialized to the memory;
- and
- resuming the isolated stateful execution session;
- wherein the given AI agent accepts structured or unstructured inputs, maintains an internal state, is associated with a Machine Learning (ML) model, and produces one or more outputs based at least in part on interactions with the ML model, the ML model having a context window, and wherein, during the execution of a given discrete computational operation, a prompt intended for the ML model is intercepted and selectively augmented to include given data that is outside the context window.
8 more elements
An apparatus, comprising:
- a hardware processor;
- and
- computing memory holding computer program instructions executed by the hardware processor, the computer program instructions configured as a runtime execution environment, the runtime execution environment configured to provide stateful agentic orchestration of a set of Artificial Intelligence (AI) agents by:
- responsive to receipt of a request associated with an AI agent workflow, wherein the request is associated with a configuration, instantiating an isolated stateful execution session;
- parsing the configuration into a set of one or more discrete computational operations for a set of AI agents;
- orchestrating execution of the set of one or more discrete computational operations within the isolated stateful execution session;
- during the execution of the set of or more discrete computational operations, detecting whether an execution boundary associated with a given AI agent has been reached;
- upon determining that the execution boundary associated with the given AI agent has been reached, suspending the isolated stateful execution session;
- serializing to a memory state data associated with the given AI agent;
- thereafter, determining whether to resume the isolated stateful execution session with the given AI agent;
- upon a determination to resume the isolated stateful execution session, re-hydrating the given AI agent, at least in part using the state data that was previously serialized to the memory;
- and
- resuming the isolated stateful execution session;
- wherein the given AI agent accepts structured or unstructured inputs, maintains an internal state, is associated with a Machine Learning (ML) model, and produces one or more outputs based at least in part on interactions with the ML model, the ML model having a context window, and wherein, during the execution of a given discrete computational operation, a prompt intended for the ML model is intercepted and selectively augmented to include given data that is outside the context window.
11 more elements
A computer program product in a non-transitory computer-readable medium comprising computer program code executable in a hardware processor, the computer program code configured as a runtime execution environment, the runtime execution environment configured to provide stateful agentic orchestration of a set of Artificial Intelligence (AI) agents by:
- responsive to receipt of a request associated with an AI agent workflow, wherein the request is associated with a configuration, instantiating an isolated stateful execution session;
- parsing the configuration into a set of one or more discrete computational operations for a set of AI agents;
- orchestrating execution of the set of one or more discrete computational operations within the isolated stateful execution session;
- during the execution of the set of or more discrete computational operations, detecting whether an execution boundary associated with a given AI agent has been reached;
- upon determining that the execution boundary associated with the given AI agent has been reached, suspending the isolated stateful execution session;
- serializing to a memory state data associated with the given AI agent;
- thereafter, determining whether to resume the isolated stateful execution session with the given AI agent;
- upon a determination to resume the isolated stateful execution session, re-hydrating the given AI agent, at least in part using the state data that was previously serialized to the memory;
- and
- resuming the isolated stateful execution session;
- wherein the given AI agent accepts structured or unstructured inputs, maintains an internal state, is associated with a Machine Learning (ML) model, and produces one or more outputs based at least in part on interactions with the ML model, the ML model having a context window, and wherein, during the execution of a given discrete computational operation, a prompt intended for the ML model is intercepted and selectively augmented to include given data that is outside the context window.
8 more elements
Dependencies are inferred from the claim text.