Adaptive Work Distribution in Distributed Systems

A computer-implemented method, comprising:

• creating, by a processor in a first node, a status object for a data processing task;
• assigning, by the processor in the first node, the data processing task to a worker of a second node;
• after a first period of time, querying, by the processor in the first node, the status object for a task status of the assigned data processing task reported by the second node, wherein the task status indicates a status of execution of the assigned data processing task by the second node;
• determining, by the processor in the first node, to reassign the assigned data processing task based on a received task status reported by the second node;
• and
• reassigning, by the processor in the first node, the assigned data processing task.

A system, comprising:

• means for creating a status object for a task;
• means for assigning the task to a worker of a system node;
• means for querying the status object for a task status of the assigned task reported by the system node after a first period of time, wherein the task status indicates a status of execution of the assigned task by the system node;
• means for determining to reassign the assigned task based on a received task status reported by the system node;
• and
• means for reassigning the assigned task.