# US 10,375,156 B2 · Using worker nodes in a distributed video encoding system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,375,156 B2
- Document ID: US10375156B2
- Status: Granted
- Kind: B2
- Application number: 14/852,467
- Filed: 2015-09-11
- Published: 2017-03-16
- Granted: 2019-08-06
- Earliest claimed: 2015-09-11
- Assignees: Facebook, Inc.
- Applicants: Facebook, Inc.
- Inventors: Michael Hamilton Coward; Amit Puntambekar
- Web page: https://patentgrep.com/patent/US10375156B2

## CPC

- H04L 67/1008: based on parameters of servers, e.g. available memory or workload (monitoring of computer activity G06F11/30)
- H04N 19/395: {involving distributed video coding [DVC], e.g. Wyner-Ziv video coding or Slepian-Wolf video coding}
- H04N 19/436: using parallelised computational arrangements
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Various of the disclosed embodiments relate to a distributed video encoding or transcoding system may utilize multiple encoding nodes to encode a video sequence by splitting the video into multiple smaller video segments. The assignment of video segments to the encoding nodes is performed to balance the use of the encoding nodes by selecting a node based on its encoding capabilities, e.g., whether the node employed a central processing unit (CPU) based encoding or a graphics processor unit (GPU) based encoding.

## Claims

### Claim 1

A computer-implemented method, implemented at a server in a distributed network, comprising:
receiving a video for encoding;
associating the video with a target encoding time;
splitting the video into multiple segments;
determining, for each of the multiple segments, one of an encoding implementation from central processing unit (CPU)-based encoding and graphics processing unit (GPU)-based encoding, wherein the determining is performed by analyzing the multiple segments using factors including target resolution and quality considerations;
queuing a number of the multiple segments for which CPU-based encoding is determined in a CPU-queue and a number of the multiple segments for which GPU-based encoding is determined in a GPU-queue, wherein the CPU-queue and the GPU-queue includes segments from at least one other video, and wherein the number of the multiple segments to be queued is dynamically adjusted based on encoding progress relative to the target encoding time;
submitting the segments from the CPU-queue and the GPU-queue to a pool of worker nodes for encoding on a priority order based on monitoring of completion speed of the video, wherein each segment is submitted to a worker node at least comprising a CPU resource capable of CPU-based encoding and a GPU resource capable of GPU-based encoding based on a run-time availability of the resource of the worker node with the corresponding encoding implementation, wherein the run-time availability is determined by comparing an available CPU resource or GPU resource with a threshold;
instructing, for each submitted segment, a corresponding resource of the worker node to encode the submitted segment based on the corresponding determined encoding implementation; and
receiving encoded segments from the worker nodes.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, further including establishing the pool of worker nodes by querying each worker node, prior to the receiving the video, for information about encoding capabilities of the worker node.

### Claim 3 (depends on 2)

The computer-implemented method of claim 2, further including receiving encoding parameters for the video and wherein the establishing the pool includes selecting worker nodes whose encoding capabilities are compatible with the encoding parameters.

### Claim 4 (depends on 2)

The computer-implemented method of claim 2, wherein the encoding capabilities further include a protocol compliance capability.

### Claim 5 (depends on 1)

The computer-implemented method of claim 1, wherein the splitting includes splitting the video such that each segment includes at least one key frame from the video that is independently decodable.

### Claim 6 (depends on 1)

The computer-implemented method of claim 1, wherein the run-time availability includes determining whether the corresponding resource of the worker node is able to meet a target time within which a segment is to be encoded.

### Claim 7 (depends on 1)

The computer-implemented method of claim 1, further comprising:
receiving encoded segments from the worker nodes; and
stitching back the encoded segments to produce an encoded video output.

### Claim 8 (depends on 7)

The computer-implemented method of claim 7, further including performing another encoding on the video at a different encoding bitrate to produce another encoded copy of the video.

### Claim 9

A non-transitory computer-readable medium storing instructions that, upon execution, cause at least one computer processor to perform a method, the instructions comprising:
instructions for receiving a video for encoding;
instructions for associating the video with a target encoding time;
instructions for splitting the video into multiple segments;
instructions for determining, for each of the multiple segments, one of an encoding implementation from central processing unit (CPU)-based encoding and graphics processing unit (GPU)-based encoding, wherein the determining is performed by analyzing the multiple segments using factors including target resolution and quality considerations;
instructions for queuing a number of the multiple segments for which CPU-based encoding is determined in a CPU-queue and a number of the multiple segments for which GPU-based encoding is determined in a GPU-queue, wherein the CPU-queue and the GPU-queue includes segments from at least one other video, and wherein the number of the multiple segments to be queued is dynamically adjusted based on encoding progress relative to the target encoding time;
instructions for submitting the segments from the CPU-queue and the GPU-queue to a pool of worker nodes for encoding on a priority order based on monitoring of completion speed of the video, wherein each segment is submitted to a worker node at least comprising a CPU resource capable of CPU-based encoding and a GPU resource capable of GPU-based encoding based on a run-time availability of the resource of the worker node with the corresponding encoding implementation, wherein the run-time availability is determined by comparing an available CPU resource or GPU resource with a threshold;
instructions for instructing, for each submitted segment, a corresponding resource of the worker node to encode the submitted segment based on the corresponding determined encoding implementation; and
instructions for receiving encoded segments from the worker nodes.

### Claim 10 (depends on 9)

The non-transitory computer-readable medium of claim 9, wherein the instructions further include instructions for establishing the pool of worker nodes by querying each worker node, prior to the receiving the video sequence, for information about encoding capabilities of the worker node.

### Claim 11 (depends on 9)

The non-transitory computer-readable medium of claim 9, wherein the instructions further include instructions for receiving encoding parameters for the video and wherein the establishing the pool includes selecting worker nodes whose encoding capabilities are compatible with the encoding parameters.

### Claim 12 (depends on 11)

The non-transitory computer-readable medium of claim 11, wherein the encoding capabilities include protocol compliance capability.

### Claim 13 (depends on 11)

The non-transitory computer-readable medium of claim 11, wherein the instructions for splitting include instructions for splitting the video such that each segment includes at least one key frame from the video that is independently decodable.

### Claim 14 (depends on 9)

The non-transitory computer-readable medium of claim 9, wherein the instructions for determining run-time availability include instructions for determining whether the corresponding resource of the worker node is able to meet a target time within which a segment is to be encoded.

### Claim 15

A computer system, comprising:
at least one processor;
at least one memory comprising instructions configured to cause the at least one processor to perform a method comprising:
receiving a video for encoding;
associating the video with a target encoding time;
splitting the video into multiple segments;
determining, for each of the multiple segments, one of an encoding implementation from central processing unit (CPU)-based encoding and graphics processing unit (GPU)-based encoding, wherein the determining is performed by analyzing the multiple segments using factors including target resolution and quality considerations;
queuing a number of the multiple segments for which CPU-based encoding is determined in a CPU-queue and a number of the multiple segments for which GPU-based encoding is determined in a GPU-queue, wherein the CPU-queue and the GPU-queue includes segments from at least one other video, and wherein the number of the multiple segments to be queued is dynamically adjusted based on encoding progress relative to the target encoding time;
submitting the segments from the CPU-queue and the GPU-queue to a pool of worker nodes for encoding on a priority order based on monitoring of completion speed of the video, wherein each segment is submitted to a worker node at least comprising a CPU resource capable of CPU-based encoding and a GPU resource capable of GPU-based encoding based on a run-time availability of the resource of the worker node with the corresponding encoding implementation, wherein the run-time availability is determined by comparing an available CPU resource or GPU resource with a threshold;
instructing, for each submitted segment, a corresponding resource of the worker node to encode the submitted segment based on the corresponding determined encoding implementation; and
receiving encoded segments from the worker nodes.

### Claim 16 (depends on 15)

The computer system of claim 15, wherein the method further includes establishing the pool of worker nodes by querying each worker node, prior to the receiving the video, for information about encoding capabilities of the worker node.

### Claim 17 (depends on 16)

The computer system of claim 16, further including receiving encoding parameters for the video and wherein the establishing the pool includes selecting worker nodes whose encoding capabilities are compatible with the encoding parameters.

### Claim 18 (depends on 16)

The computer system of claim 16, wherein the encoding capabilities include protocol compliance capability.

### Claim 19 (depends on 15)

The computer system of claim 15, wherein the splitting includes splitting the video such that each segment includes at least one key frame from the video that is independently decodable.

### Claim 20 (depends on 15)

The computer system of claim 15, wherein the run-time availability includes determining whether the corresponding resource of the worker node is able to meet a target time within which a segment is to be encoded.