# US 12,113,878 B2 · Methods and systems for scheduling a workflow

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,113,878 B2
- Document ID: US12113878B2
- Status: Granted
- Kind: B2
- Application number: 17/704,688
- Filed: 2022-03-25
- Published: 2023-01-19
- Granted: 2024-10-08
- Earliest claimed: 2021-07-06
- Assignees: Tencent America LLC
- Applicants: Tencent America LLC
- Inventors: Iraj Sodagar
- Web page: https://patentgrep.com/patent/US12113878B2

## CPC

- H04L 67/62: Establishing a time schedule for servicing the requests
- G06F 9/5038: {considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration (scheduling strategies G06F9/4881 and subgroups)}
- H04L 67/10: in which an application is distributed across nodes in the network (software deployment G06F8/60; multiprogramming arrangements G06F9/46)
- H04L 69/24: Negotiation of communication capabilities
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Systems and methods for scheduling a network-based media processing (NBMP) workflow based on a scheme mode are provided. The scheme mode may be obtained by a NBMP workflow manager based on received information that indicates the scheme mode. The NBMP workflow manager may schedule tasks of the NBMP workflow to be executed to process media in accordance with the scheme mode indicated in the information. The scheme mode may be a by order mode, a by duration mode, a by event mode, or a by timeslot mode.

## Claims

### Claim 1

A method performed by at least one processor that implements a network-based media processing (NBMP) workflow manager, the method comprising:
sending, to an NBMP client, first information indicating that the NBMP workflow manager supports a scheme mode for scheduling an NBMP workflow;
receiving, from the NBMP client, second information indicating the scheme mode for scheduling the NBMP workflow;
obtaining the scheme mode based on the second information;
scheduling tasks of the NBMP workflow to be executed to process media in accordance with the scheme mode indicated in the second information,
wherein the scheme mode indicated in the second information and indicated to be supported by the NBMP workflow manager in the first information is:
a first scheme mode in which each task of the NBMP workflow is executed once for an entirety of an input of the task,
a second scheme mode in which each task of the NBMP workflow is executed for a predetermined duration of the input of the task or for a predetermined duration of an output of the task,
a third scheme mode in which each task of the NBMP workflow is started based on a start event or stopped based on a stop event, or
a fourth scheme mode in which each task of the NBMP workflow is started according to a predetermined schedule, wherein the predetermined schedule includes time slots for executing the tasks; and
performing the scheduled tasks of the NBMP workflow, wherein
the scheme mode indicated in the second information, and indicated to be supported by the NBMP workflow manager in the first information, is the first scheme mode,
the receiving the second information comprises, receiving, from the NBMP client, a syntax element with a first value indicating the first scheme mode for scheduling a first plurality of tasks of the NBMP workflow, the first value being the second information,
the method further comprises:
receiving, from the NBMP client, the syntax element with a second value indicating the second scheme mode for scheduling a second plurality of tasks of the NBMP workflow;
scheduling the second plurality of tasks of the NBMP workflow to be executed to process media in accordance with the second scheme mode indicated by the second value;
receiving, from the NBMP client, the syntax element with a third value indicating the third scheme mode for scheduling a third plurality of tasks of the NBMP workflow;
scheduling the third plurality of tasks of the NBMP workflow to be executed to process media in accordance with the third scheme mode indicated by the third value;
receiving, from the NBMP client, the syntax element with a fourth value indicating the fourth scheme mode for scheduling a fourth plurality of tasks of the NBMP workflow;
scheduling the fourth plurality of tasks of the NBMP workflow to be executed to process media in accordance with the fourth scheme mode indicated by the fourth value, and
the first value, the second value, the third value, and the fourth value of the syntax element are different values from each other.

### Claim 2 (depends on 1)

The method of claim 1, wherein
the sending comprises sending, to the NBMP client, the first information that indicates that the NBMP workflow manager supports the scheme mode, based on receiving a workflow description document (WDD) from the NBMP client that includes a parameter that has a value that indicates that capabilities of the NBMP workflow manager are requested.

### Claim 3 (depends on 1)

The method of claim 1, wherein
the method further comprises scheduling the NBMP workflow based on receiving a workflow description document (WDD) that includes third information indicating the NBMP workflow, or
the method further comprises scheduling a group of tasks based on receiving a task group object that indicates the group of tasks, or
the scheduling the tasks comprises scheduling a task based on receiving a task description document (TDD) that indicates the task.

### Claim 4 (depends on 1)

The method of claim 1, further comprises:
sending, to the NBMP client, third information indicating whether the NBMP workflow manager can schedule the NBMP workflow, a group of tasks, or a task, based on receiving a parameter from the NBMP client that has a value that indicates that the NBMP client requests to investigate whether the NBMP workflow manager can accommodate scheduling of the NBMP workflow, the group of tasks, or the task.

### Claim 5 (depends on 1)

The method of claim 1, wherein
the scheme mode indicated in the second information, and indicated to be supported by the NBMP workflow manager in the first information, is the third scheme mode.

### Claim 6 (depends on 5)

The method of claim 5, further comprising:
obtaining the start event or the stop event by:
receiving additional information indicating the start event or the stop event, or
generating the start event or the stop event,
wherein the scheduling the tasks of the NBMP workflow comprises scheduling at least one of the tasks of the NBMP workflow to be started based on the start event that is obtained or stopped based on the stop event that is obtained.

### Claim 7 (depends on 5)

The method of claim 5, wherein the scheduling the tasks of the NBMP workflow comprises changing a state of at least one of the tasks of the NBMP workflow to a paused state.

### Claim 8 (depends on 1)

The method of claim 1, wherein
the scheme mode indicated in the second information, and indicated to be supported by the NBMP workflow manager in the first information, is the fourth scheme mode.

### Claim 9 (depends on 8)

The method of claim 8, further comprising receiving additional information indicating a scheduling mode of at least one task of the NBMP workflow,
wherein the scheduling mode is one from among:
a first scheduling mode that indicates that a timeslot of the at least one task within the predetermined schedule is explicitly signaled to the NBMP workflow manager, or
a second scheduling mode that indicates that the NBMP workflow manager obtains the timeslot through inference.

### Claim 10

A system comprising:
at least one memory configured to store computer program code; and
at least one processor configured to access the computer program code and operate as instructed by the computer program code, the computer program code comprising:
sending code configured to cause a network-based media processing (NBMP) workflow manager, implemented by the at least one processor, to send first information indicating that the NBMP workflow manager supports a scheme mode for scheduling an NBMP workflow;
obtaining code configured to cause the NBMP workflow manager to obtain the scheme mode for scheduling the NBMP workflow, the scheme mode obtained based on received second information, from the NBMP client, that indicates the scheme mode; and
scheduling code configured to cause the NBMP workflow manager to schedule tasks of the NBMP workflow to be executed to process media in accordance with the scheme mode indicated in the received second information,
wherein the scheme mode indicated in the received second information and indicated to be supported by the NBMP workflow manager in the first information is:
a first scheme mode in which each task of the NBMP workflow is executed once for an entirety of an input of the task,
a second scheme mode in which each task of the NBMP workflow is executed for a predetermined duration of the input of the task or for a predetermined duration of an output of the task,
a third scheme mode in which each task of the NBMP workflow is started based on a start event or stopped based on a stop event, or
a fourth scheme mode in which each task of the NBMP workflow is started according to a predetermined schedule, wherein the predetermined schedule includes time slots for executing the tasks, wherein
the scheme mode indicated in the second information, and indicated to be supported by the NBMP workflow manager in the first information, is the first scheme mode,
the receiving the second information comprises, receiving, from the NBMP client, a syntax element with a first value indicating the first scheme mode for scheduling a first plurality of tasks of the NBMP workflow, the first value being the second information,
the processor is further configured to access the computer program code and operate as instructed by the computer program code to implement:
receiving, from the NBMP client, the syntax element with a second value indicating the second scheme mode for scheduling a second plurality of tasks of the NBMP workflow;
scheduling the second plurality of tasks of the NBMP workflow to be executed to process media in accordance with the second scheme mode indicated by the second value;
receiving, from the NBMP client, the syntax element with a third value indicating the third scheme mode for scheduling a third plurality of tasks of the NBMP workflow;
scheduling the third plurality of tasks of the NBMP workflow to be executed to process media in accordance with the third scheme mode indicated by the third value;
receiving, from the NBMP client, the syntax element with a fourth value indicating the fourth scheme mode for scheduling a fourth plurality of tasks of the NBMP workflow;
scheduling the fourth plurality of tasks of the NBMP workflow to be executed to process media in accordance with the fourth scheme mode indicated by the fourth value, and
the first value, the second value, the third value, and the fourth value of the syntax element are different values from each other.

### Claim 11 (depends on 10)

The system of claim 10, wherein
the sending code is further configured to cause the NBMP workflow manager to send, to the NBMP client, the first information that indicates that the NBMP workflow manager supports the scheme mode, based on receiving a workflow description document (WDD) from the NBMP client that includes a parameter that has a value that indicates that capabilities of the NBMP workflow manager are requested.

### Claim 12 (depends on 10)

The system of claim 10, wherein the scheduling code is further configured to cause the NBMP workflow manager to:
schedule the NBMP workflow based on receiving a workflow description document (WDD) that includes third information indicating the NBMP workflow; or
schedule a group of tasks based on receiving a task group object that indicates the group of tasks; or
schedule a task based on receiving a task description document (TDD) that indicates the task.

### Claim 13 (depends on 10)

The system of claim 10, wherein the computer program code further comprises:
response code configured to cause the NBMP workflow manager to send, to the NBMP client, third information indicating whether the NBMP workflow manager can schedule the NBMP workflow, a group of tasks, or a task, based on receiving a parameter from the NBMP client that has a value that indicates that the NBMP client requests to investigate whether the NBMP workflow manager can accommodate scheduling of the NBMP workflow, the group of tasks, or the task.

### Claim 14 (depends on 10)

The system of claim 10, wherein
the scheme mode indicated in the received second information, and indicated to be supported by the NBMP workflow manager in the first information, is the third scheme mode.

### Claim 15 (depends on 14)

The system of claim 14, wherein the start event or the stop event is obtained by the NBMP workflow manager.

### Claim 16 (depends on 14)

The system of claim 14, wherein the scheduling code is further configured to cause the NBMP workflow manager to change a state of at least one of the tasks of the NBMP workflow to a paused state.

### Claim 17 (depends on 10)

The system of claim 10, wherein
the scheme mode indicated in the received second information, and indicated to be supported by the NBMP workflow manager in the first information, is the fourth scheme mode.

### Claim 18 (depends on 17)

The system of claim 17, wherein the scheduling code is further configured to cause the NBMP workflow manager to schedule at least one task of the NBMP workflow to be executed in accordance with a scheduling mode that is indicated by additional information received by the NBMP workflow manager,
wherein the scheduling mode is one from among:
a first scheduling mode that indicates that a timeslot of the at least one task within the predetermined schedule is explicitly signaled to the NBMP workflow manager, or
a second scheduling mode that indicates that the NBMP workflow manager obtains the timeslot through inference.

### Claim 19

A non-transitory computer-readable medium storing computer code that is configured to, when executed by at least one processor, cause the at least one processor to implement a network-based media processing (NBMP) workflow manager that:
sends, to an NBMP client, first information indicating that the NBMP workflow manager supports a scheme mode for scheduling an NBMP workflow;
obtains the scheme mode for scheduling the NBMP workflow, the scheme mode obtained based on received second information, from the NBMP client, that indicates the scheme mode; and
schedules tasks of the NBMP workflow to be executed to process media in accordance with the scheme mode indicated in the received second information,
wherein the scheme mode indicated in the received second information and indicated to be supported by the NBMP workflow manager in the first information is:
a first scheme mode in which each task of the NBMP workflow is executed once for an entirety of an input of the task,
a second scheme mode in which each task of the NBMP workflow is executed for a predetermined duration of the input of the task or for a predetermined duration of an output of the task,
a third scheme mode in which each task of the NBMP workflow is started based on a start event or stopped based on a stop event, or
a fourth scheme mode in which each task of the NBMP workflow is started according to a predetermined schedule, wherein the predetermined schedule includes time slots for executing the tasks, wherein
the scheme mode indicated in the second information, and indicated to be supported by the NBMP workflow manager in the first information, is the first scheme mode,
the receiving the second information comprises, receiving, from the NBMP client, a syntax element with a first value indicating the first scheme mode for scheduling a first plurality of tasks of the NBMP workflow, the first value being the second information,
the computer code that is configured to, when executed by at least one processor, cause the at least one processor to implement:
receiving, from the NBMP client, the syntax element with a second value indicating the second scheme mode for scheduling a second plurality of tasks of the NBMP workflow;
scheduling the second plurality of tasks of the NBMP workflow to be executed to process media in accordance with the second scheme mode indicated by the second value;
receiving, from the NBMP client, the syntax element with a third value indicating the third scheme mode for scheduling a third plurality of tasks of the NBMP workflow;
scheduling the third plurality of tasks of the NBMP workflow to be executed to process media in accordance with the third scheme mode indicated by the third value;
receiving, from the NBMP client, the syntax element with a fourth value indicating the fourth scheme mode for scheduling a fourth plurality of tasks of the NBMP workflow;
scheduling the fourth plurality of tasks of the NBMP workflow to be executed to process media in accordance with the fourth scheme mode indicated by the fourth value, and
the first value, the second value, the third value, and the fourth value of the syntax element are different values from each other.