# US 2025/0165281 A1 · Dynamically Adapting Task Execution Parallelism of Distributed Applications

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0165281 A1
- Document ID: US20250165281A1
- Status: Published application
- Kind: A1
- Application number: 18/515,926
- Filed: 2023-11-21
- Published: 2025-05-22
- Earliest claimed: 2023-11-21
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: Michael Kaufmann; Radu Ioan Stoica; Pascal Spörri; Torsten Steinbach; Shrinivas Kulkarni
- Web page: https://patentgrep.com/patent/US20250165281A1

## CPC

- G06F 9/485: {Task life-cycle, e.g. stopping, restarting, resuming execution (G06F9/4881 takes precedence)}
- G06F 9/5066: {Algorithms for mapping a plurality of inter-dependent sub-tasks onto a plurality of physical CPUs (mappping at compile time, see G06F8/451)}
- G06F 9/4881: {Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues}
- G06F 9/505: {considering the load}
- G06N 20/00: Machine learning
- G06F 2209/483: Multiproc
- G06F 2209/5017: Task decomposition
- G06F 2209/5018: Thread allocation
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Scheduling tasks for concurrent execution in parallel is provided. Metrics corresponding to execution of tasks in a plurality of stages of a distributed application are received from a set of worker nodes. An analysis of the metrics corresponding to the execution of the tasks in the plurality of stages of the distributed application is performed using a prediction model. An optimal per-stage task execution parallelism level for each respective stage of the plurality of stages is determined using the prediction model based on the analysis of the metrics. The tasks are scheduled to execute on the set of worker nodes based on the optimal per-stage task execution parallelism level determined for each respective stage of the plurality of stages to improve at least one of runtime of the distributed application and resource utilization by the distributed application.

## Claims

### Claim 1

A computer-implemented method for scheduling tasks for concurrent execution in parallel, the computer-implemented method comprising:
receiving, by a computer, from a set of worker nodes via a network, metrics corresponding to execution of tasks in a plurality of stages of a distributed application that are running on the set of worker nodes;
performing, by the computer, using a prediction model, an analysis of the metrics corresponding to the execution of the tasks in the plurality of stages of the distributed application received from the set of worker nodes;
determining, by the computer, using the prediction model, an optimal per-stage task execution parallelism level for each respective stage of the plurality of stages based on the analysis of the metrics corresponding to the execution of the tasks in the plurality of stages of the distributed application; and
scheduling, by the computer, the tasks to execute on the set of worker nodes based on the optimal per-stage task execution parallelism level determined for each respective stage of the plurality of stages to improve at least one of runtime of the distributed application and resource utilization by the distributed application.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, further comprising:
running, by the computer, a set of artificial workloads on the distributed application;
generating, by the computer, generic metrics corresponding to execution of a set of tasks in each respective stage of the plurality of stages of the distributed application based on running the artificial workloads on the distributed application; and
generating, by the computer, a fingerprint for each respective stage of the plurality of stages of the distributed application based on the generic metrics corresponding to the execution of the set of tasks in each respective stage of the plurality of stages of the distributed application.

### Claim 3 (depends on 2)

The computer-implemented method of claim 2, further comprising:
training, by the computer, the prediction model initially offline to form a trained prediction model using the fingerprint of each respective stage of the plurality of stages of the distributed application that was based on the generic metrics corresponding to the execution of the set of tasks in each respective stage of the plurality of stages; and
determining, by the computer, using the trained prediction model, the optimal per-stage task execution parallelism level of each respective stage of the plurality of stages of the distributed application for a first run of the distributed application.

### Claim 4 (depends on 3)

The computer-implemented method of claim 3, further comprising:
scheduling, by the computer, the set of tasks of each respective stage of the plurality of stages of the distributed application to execute on the set of worker nodes during the first run of the distributed application based on the optimal per-stage task execution parallelism level determined for each respective stage of the plurality of stages; and
collecting, by the computer, the metrics corresponding to execution of the set of tasks in each respective stage of the plurality of stages of the distributed application from the set of worker nodes during the first run of the distributed application.

### Claim 5 (depends on 4)

The computer-implemented method of claim 4, further comprising:
updating, by the computer, using the trained prediction model, the optimal per-stage task execution parallelism level of each respective stage of the plurality of stages of the distributed application after the first run of the distributed application to form an updated optimal per-stage task execution parallelism level based on the metrics corresponding to the execution of the set of tasks in each respective stage of the plurality of stages of the distributed application collected from the set of worker nodes during the first run of the distributed application; and
utilizing, by the computer, the updated optimal per-stage task execution parallelism level to schedule the set of tasks in each respective stage of the plurality of stages of the distributed application on the set of worker nodes during subsequent runs of the distributed application.

### Claim 6 (depends on 5)

The computer-implemented method of claim 5, further comprising:
collecting, by the computer, historic metrics during at least one of the subsequent runs of the distributed application and runs of a set of similar distributed applications; and
performing, by the computer, using the trained prediction model, a subsequent update of the updated optimal per-stage task execution parallelism level of each respective stage of the plurality of stages of the distributed application based on the historic metrics collected during the at least one of the subsequent runs of the distributed application and the runs of the set of similar distributed applications.

### Claim 7 (depends on 6)

The computer-implemented method of claim 6, wherein the computer determines similarity between the distributed application and the set of similar distributed applications by comparing one or more of anonymized execution graphs, data dependencies, and binaries corresponding to the distributed application and each of the set of similar distributed applications.

### Claim 8 (depends on 1)

The computer-implemented method of claim 1, wherein the computer schedules a set of tasks of a same stage on a same worker node to reduce cold starts, and wherein the computer schedules tasks of two or more concurrent stages in parallel such that each of the two or more concurrent stages are estimated to complete at a same time avoiding straggler stage delays.

### Claim 9 (depends on 1)

The computer-implemented method of claim 1, wherein the metrics include at least one of a number of tasks in a stage, average runtime of each task in the stage, runtime variance of each task in the stage based on level of correlation to input data size, task runtime on warm executors versus task runtime on cold executors, amount of input and output data read or written by each task in the stage, amount of intermediary data read or written by each task in the stage, amount of memory consumed, amount of data spilled to secondary storage, amount of input and output variance by each task in the stage, and time needed to load shared data from a shared data cache.

### Claim 10

A computer system for scheduling tasks for concurrent execution in parallel, the computer system comprising:
a communication fabric;
a storage device connected to the communication fabric, wherein the storage device stores program instructions; and
a processor connected to the communication fabric, wherein the processor executes the program instructions to:
receive, from a set of worker nodes via a network, metrics corresponding to execution of tasks in a plurality of stages of a distributed application that are running on the set of worker nodes;
perform, using a prediction model, an analysis of the metrics corresponding to the execution of the tasks in the plurality of stages of the distributed application received from the set of worker nodes;
determine, using the prediction model, an optimal per-stage task execution parallelism level for each respective stage of the plurality of stages based on the analysis of the metrics corresponding to the execution of the tasks in the plurality of stages of the distributed application; and
schedule the tasks to execute on the set of worker nodes based on the optimal per-stage task execution parallelism level determined for each respective stage of the plurality of stages to improve at least one of runtime of the distributed application and resource utilization by the distributed application.

### Claim 11 (depends on 10)

The computer system of claim 10, wherein the processor further executes the program instructions to:
run a set of artificial workloads on the distributed application;
generate generic metrics corresponding to execution of a set of tasks in each respective stage of the plurality of stages of the distributed application based on running the artificial workloads on the distributed application; and
generate a fingerprint for each respective stage of the plurality of stages of the distributed application based on the generic metrics corresponding to the execution of the set of tasks in each respective stage of the plurality of stages of the distributed application.

### Claim 12 (depends on 11)

The computer system of claim 11, wherein the processor further executes the program instructions to:
train the prediction model initially offline to form a trained prediction model using the fingerprint of each respective stage of the plurality of stages of the distributed application that was based on the generic metrics corresponding to the execution of the set of tasks in each respective stage of the plurality of stages; and
determine, using the trained prediction model, the optimal per-stage task execution parallelism level of each respective stage of the plurality of stages of the distributed application for a first run of the distributed application.

### Claim 13 (depends on 12)

The computer system of claim 12, wherein the processor further executes the program instructions to:
schedule the set of tasks of each respective stage of the plurality of stages of the distributed application to execute on the set of worker nodes during the first run of the distributed application based on the optimal per-stage task execution parallelism level determined for each respective stage of the plurality of stages; and
collect the metrics corresponding to execution of the set of tasks in each respective stage of the plurality of stages of the distributed application from the set of worker nodes during the first run of the distributed application.

### Claim 14 (depends on 13)

The computer system of claim 13, wherein the processor further executes the program instructions to:
update, using the trained prediction model, the optimal per-stage task execution parallelism level of each respective stage of the plurality of stages of the distributed application after the first run of the distributed application to form an updated optimal per-stage task execution parallelism level based on the metrics corresponding to the execution of the set of tasks in each respective stage of the plurality of stages of the distributed application collected from the set of worker nodes during the first run of the distributed application; and
utilize the updated optimal per-stage task execution parallelism level to schedule the set of tasks in each respective stage of the plurality of stages of the distributed application on the set of worker nodes during subsequent runs of the distributed application.

### Claim 15

A computer program product for scheduling tasks for concurrent execution in parallel, the computer program product comprising a computer-readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to:
receive, from a set of worker nodes via a network, metrics corresponding to execution of tasks in a plurality of stages of a distributed application that are running on the set of worker nodes;
perform, using a prediction model, an analysis of the metrics corresponding to the execution of the tasks in the plurality of stages of the distributed application received from the set of worker nodes;
determine, using the prediction model, an optimal per-stage task execution parallelism level for each respective stage of the plurality of stages based on the analysis of the metrics corresponding to the execution of the tasks in the plurality of stages of the distributed application; and
schedule the tasks to execute on the set of worker nodes based on the optimal per-stage task execution parallelism level determined for each respective stage of the plurality of stages to improve at least one of runtime of the distributed application and resource utilization by the distributed application.

### Claim 16 (depends on 15)

The computer program product of claim 15, wherein the program instructions further cause the computer to:
run a set of artificial workloads on the distributed application;
generate generic metrics corresponding to execution of a set of tasks in each respective stage of the plurality of stages of the distributed application based on running the artificial workloads on the distributed application; and
generate a fingerprint for each respective stage of the plurality of stages of the distributed application based on the generic metrics corresponding to the execution of the set of tasks in each respective stage of the plurality of stages of the distributed application.

### Claim 17 (depends on 16)

The computer program product of claim 16, wherein the program instructions further cause the computer to:
train the prediction model initially offline to form a trained prediction model using the fingerprint of each respective stage of the plurality of stages of the distributed application that was based on the generic metrics corresponding to the execution of the set of tasks in each respective stage of the plurality of stages; and
determine, using the trained prediction model, the optimal per-stage task execution parallelism level of each respective stage of the plurality of stages of the distributed application for a first run of the distributed application.

### Claim 18 (depends on 17)

The computer program product of claim 17, wherein the program instructions further cause the computer to:
schedule the set of tasks of each respective stage of the plurality of stages of the distributed application to execute on the set of worker nodes during the first run of the distributed application based on the optimal per-stage task execution parallelism level determined for each respective stage of the plurality of stages; and
collect the metrics corresponding to execution of the set of tasks in each respective stage of the plurality of stages of the distributed application from the set of worker nodes during the first run of the distributed application.

### Claim 19 (depends on 18)

The computer program product of claim 18, wherein the program instructions further cause the computer to:
update, using the trained prediction model, the optimal per-stage task execution parallelism level of each respective stage of the plurality of stages of the distributed application after the first run of the distributed application to form an updated optimal per-stage task execution parallelism level based on the metrics corresponding to the execution of the set of tasks in each respective stage of the plurality of stages of the distributed application collected from the set of worker nodes during the first run of the distributed application; and
utilize the updated optimal per-stage task execution parallelism level to schedule the set of tasks in each respective stage of the plurality of stages of the distributed application on the set of worker nodes during subsequent runs of the distributed application.

### Claim 20 (depends on 19)

The computer program product of claim 19, wherein the program instructions further cause the computer to:
collect historic metrics during at least one of the subsequent runs of the distributed application and runs of a set of similar distributed applications; and
perform, using the trained prediction model, a subsequent update of the updated optimal per-stage task execution parallelism level of each respective stage of the plurality of stages of the distributed application based on the historic metrics collected during the at least one of the subsequent runs of the distributed application and the runs of the set of similar distributed applications.
