Affinity-based routing and execution for workflow service
AssigneeAmazon Technologies, Inc.
InventorKuldeep Gupta
Life of this patent
Estimating the term…
Abstract
A workflow service implements policies to leverage affinities to improve system performance. Example types of policies include a parent activity affinity (e.g., data for a task already exists locally to a node), a resource group affinity (e.g., a task requires a particular type of resource capability), or a code affinity (e.g., necessary code for the task is already stored locally to a node). A node for a particular task is selected based on an affinity policy or policy statement (e.g., the type indicated in a policy of a workflow definition) and configuration information, and the task routed to the selected node.
Claims20
A system, comprising:
- a plurality of computers configured to implement a fleet of workflow activity execution nodes;
- and
- one or computers configured to implement a workflow engine configured to manage execution of workflows according to workflow definitions that specify a flow of a plurality of different tasks and one or more affinity policies for one or more of the different tasks, wherein the workflow engine comprises:
- an execution node tracker implemented via one or more processors and memory and configured to track configuration information for workflow activity execution nodes, including:
- identifiers of task execution code locally stored on individual workflow activity execution nodes, and
- resource capability information for individual workflow activity execution nodes;
- and
- a task router implemented via one or more processors and memory and configured to:
- identify, for a task of a workflow, an affinity policy from a workflow definition that includes the task;
- select, based at least in part on the affinity policy and the configuration information from the execution node tracker, one of the workflow activity execution nodes to execute the task, where the affinity policy indicates one or more of:
- a parent activity affinity that indicates that the task should be routed to a workflow activity execution node that executed a specified previous task of the workflow,
- a resource group affinity that indicates that the task should be routed to a workflow activity execution node having a specified resource capability, or
- a code affinity that indicates that the task should be routed to a workflow activity execution node having code for a specified task execution code identifier;
- and
- route the task to the selected workflow activity execution node.
A method, comprising:
- identifying, for a task of a workflow, a policy from a workflow definition including the task, the policy indicating one or more of:
- a parent activity affinity indicating that the task should be routed to an activity execution node that executed a specified previous task of the workflow,
- a resource group affinity indicating that the task should be routed to an activity execution node having a specified resource capability, or
- a code affinity indicating that the task should be routed to an activity execution node having code for a specified task execution code identifier;
- selecting one of a fleet of activity execution nodes to execute the task based at least in part on the affinity type indicated by the policy, and configuration information comprising:
- identifiers of task execution code locally stored on individual activity execution nodes, and
- information for individual activity execution nodes that indicates resource capability;
- and
- routing the task to the selected activity execution node.
One or more non-transitory, computer-readable storage media storing program instructions that when executed on or across one or more hardware processors configure a workflow service to perform:
- identifying, for a task of a workflow, a policy from a workflow definition including the task, the policy indicating one or more of:
- a parent activity affinity indicating that the task should be routed to an activity execution node that executed a specified previous task of the workflow,
- a resource group affinity indicating that the task should be routed to an activity execution node having a specified resource capability, or
- a code affinity indicating that the task should be routed to an activity execution node having code for a specified task execution code identifier;
- selecting one of a fleet of activity execution nodes to execute the task based at least in part on the affinity type indicated by the policy, and configuration information comprising:
- identifiers of task execution code locally stored on individual activity execution nodes, and
- information for individual activity execution nodes that indicates resource capability;
- and
- routing the task to the selected activity execution node.
Dependencies are inferred from the claim text.