US 2023/0145846 A1 · SYSTEMS AND METHODS FOR AFFINITY-BASED DISTRIBUTED WORK POOL SCHEDULING

Abstract

Systems and methods for affinity-based distribution of work are disclosed. The method may include determining a classification for a plurality of tasks. The method may further include classifying a compute cost and an affinity for each task of the plurality of tasks. The method may further include assigning the plurality of tasks to a work collection. The method may further include selecting one or more tasks from the plurality of tasks in the work collection based on a selection strategy of a worker node. The method may further include performing a compute task associated with the one or more tasks. The method may further include generating a set of results based on performing the compute task.

Claims19

Dependent claims

A method of distributing tasks, the method comprising:

determining a classification, by a producer node, for a plurality of tasks, wherein the classification comprises classifying, for each task of the plurality of tasks, a compute cost and an affinity to a different task of the plurality of tasks;
assigning, by the producer node, the plurality of tasks to a work collection;
selecting, by a worker node, one or more tasks from the plurality of tasks in the work collection based on a selection strategy of the worker node, wherein the selection strategy is based on a target affinity and a target cost of the worker node;
performing, by the worker node, a compute task associated with the one or more tasks of the plurality of tasks;
and
generating, by the worker node, a set of results based on performing the compute task.

3 more elements

A cloud computing environment for distributed workflow management, the cloud computing environment comprising:

a producer node configured to:
determine a classification for a plurality of tasks, wherein the classification comprises a compute cost and an affinity, for each task of the plurality of task, to a different task of the plurality of tasks;
and
assign the plurality of tasks to a work collection;
a worker node configured to:
select one or more tasks from the plurality of tasks in the work collection based on a selection strategy of the worker node, wherein the selection strategy is based on a target affinity and a target cost of the worker node;
perform a compute task associated with the one or more tasks of the plurality of tasks;
and
generate a set of results based on performing the compute task;
a backlog supervisor configured to:
monitor the worker node and the work collection;
determine, based on the monitoring, a number of additional worker nodes to activate;
and
activate the number of additional worker nodes;
a lifetime supervisor configured to:
monitor the worker node and the work collection;
determine, based on the monitoring, a failure state of the worker node and a failed status of the one or more tasks of the plurality of tasks;
and
select, by a second worker node, the one or more tasks of the plurality of tasks based on the failed status of the one or more tasks.

16 more elements

Dependencies are inferred from the claim text.

Systems and methods for affinity-based distributed work pool scheduling

Life of this patent

Systems and methods for affinity-based distributed work pool scheduling

Life of this patent

Abstract

Claims19