Network bandwidth sharing in a distributed computing system
AssigneeFacebook, Inc.
InventorJian Li
Life of this patent
Estimating the term…
Abstract
A bandwidth sharing system is provided that has worker nodes with executor threads for copying data from source nodes to destination nodes based on assigned copy jobs. To execute its assigned copy job, a worker thread of a worker node registers its copy job with the source node. The source node allocates a portion of its bandwidth to the copy job. The source node sends the data of the copy job to the worker thread, and the worker thread forwards the data to the destination node. Upon completion of the copy job, the worker thread deregisters the copy job. The deregistration allows the source node to reallocate the portion of its bandwidth that was allocated to the copy job to another copy job.
Claims20
A method performed by a source node for sharing of network bandwidth, the method comprising:
- receiving, from a global traffic manager node at the source node, bandwidth allocation for multiple types of copy jobs executing in a distributed computing system for moving data between multiple source nodes and multiple destination nodes;
- upon receiving, at the source node and from a worker node, a registration of a copy job for copying data from the source node to a destination node, allocating bandwidth of the source node among copy jobs that are registered in accordance with a type of each of the copy jobs and the bandwidth allocation received from the global traffic manager node for each of the types of the copy jobs;
- and
- executing, by the worker node, the copy job at the source node to copy the data in accordance with bandwidth allocated to the copy job to the worker node for forwarding to the destination node, wherein as copy jobs are registered and deregistered the network bandwidth is reallocated among the copy jobs.
A data center comprising:
- multiple worker nodes for copying data from source nodes in one data center to destination nodes in another data center, each worker node having multiple executor threads, wherein each executor thread is assigned an entire copy job or a portion of the copy job, each copy job specifying a source node, a destination node, and data to be copied, each executor thread adapted to register its copy job with the source node before starting the copying of data and deregister its copy job from the source node after completing the copying of data;
- multiple data nodes, some of the multiple data nodes being source nodes of copy jobs and some of the multiple data nodes being destination nodes of copy jobs, wherein a source node is adapted to:
- receive copy jobs and bandwidth allocation for different types of copy jobs;
- upon receiving a registration or deregistration of a copy job from an executor thread, allocate bandwidth of the source node among the copy jobs that are registered at the source node in accordance with a type of each of the copy jobs and the bandwidth allocation received for each of the types of the copy jobs;
- and
- send data in accordance with the bandwidth allocated to the copy job to the executor thread of the copy job for forwarding to the destination node of the copy job;
- and
- a global traffic manager node for distributing the bandwidth allocation for different types of the copy jobs to each of the source nodes.
A computer-readable storage medium for storing computer-executable instructions for controlling a source node for sharing of network bandwidth, the computer-executable instructions comprising instructions adapted to:
- receive bandwidth allocation for different types of copy jobs executing in a distributed computing system to copy data between multiple source nodes and multiple destination nodes, each copy job specifying a source node, a destination node, and data to copy;
- receive registrations and deregistrations of copy jobs from executor threads executing on worker nodes, each executor thread being assigned a copy job;
- and
- upon receiving a registration of a copy job,
- reallocate bandwidth of the source node among copy jobs registered at the source node factoring a type of each of the copy jobs registered at the source node and the bandwidth allocation provided by a global traffic manager node for different types of copy jobs;
- send data to be copied by the copy job to the executor thread of the copy job;
- and
- upon receiving a deregistration of a copy job, reallocate bandwidth of the source node among remaining of the copy jobs registered at the source node.
Dependencies are inferred from the claim text.