# US 2015/0103645 A1 · Controller side method of generating and updating a controller assignment list

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2015/0103645 A1
- Document ID: US20150103645A1
- Status: Published application
- Kind: A1
- Application number: 14/070,448
- Filed: 2013-11-01
- Published: 2015-04-16
- Earliest claimed: 2013-10-10
- Assignees: VMware, Inc.
- Applicants: VMware, Inc.
- Inventors: Jianjun Shen; Hua Wang; James Joseph Stabile; Xuan Zhang
- Web page: https://patentgrep.com/patent/US20150103645A1

## CPC

- H04L 41/042: comprising distributed management centres cooperatively managing the network
- G06F 9/452: {Remote windowing, e.g. X-Window System, desktop virtualisation (protocols for virtual reality H04L67/131)}
- H04L 41/344: Out-of-band transfers
- H04L 45/64: using an overlay routing layer
- H04L 47/125: by balancing the load, e.g. traffic engineering
- H04L 67/131: Protocols for games, networked simulations or virtual reality
- H04L 41/0659: by isolating or reconfiguring faulty entities
- H04L 61/103: across network layers, e.g. resolution of network layer into physical layer addresses or address resolution protocol [ARP]
- H04L 61/5014: using dynamic host configuration protocol [DHCP] or bootstrap protocol [BOOTP]
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Some embodiments provide a novel method for distributing control-channel communication load between multiple controllers in a network control system. In some embodiments, the controllers manage physical forwarding elements that forward data between several computing devices (also called hosts or host computers), some or all of which execute one or more virtual machines (VMs). The method of some embodiments distributes a controller assignment list to the host computers. The host computers use this list to identify the controllers with which they need to interact to perform some of the forwarding operations of their associated logical forwarding elements. In some embodiments, agents executing on the host computers (1) review the controller assignment list to identify the appropriate controllers, and (2) establish control channel communications with these controllers to obtain the needed data for effectuating the forwarding operations of their associated physical forwarding elements. These agents in some embodiments are responsible for out-of-band control channel communication with the controllers.

## Claims

### Claim 1

For a network control system that includes a plurality of host computers that execute a plurality of physical forwarding elements (PFEs) that collectively implement a plurality of logical forwarding elements (LFEs), a method of managing the LFEs comprising:
selecting a common range of index values for different types of LFEs;
identifying a plurality of controllers to manage the LFEs;
generating a controller assignment list that associates each controller with at least one index value in the common index value range, wherein the index values are derivable from identifiers of the LFEs;
distributing the controller assignment list so that the list is received by the hosts, said list for the hosts to use to identify which controllers to request data necessary for effectuating a subset of the operations of the LFEs.

### Claim 2 (depends on 1)

The method of claim 1, wherein the controller assignment list associates each controller with a range of index value ranges.

### Claim 3 (depends on 1)

The method of claim 1, wherein when first and second index values that are derivable from first and second identifiers of first and second LFEs fall within a particular range of index values, a controller that is associated with the particular index value range manages both the first and second LFEs.

### Claim 4 (depends on 3)

The method of claim 3, wherein the first and second LFEs have the same type.

### Claim 5 (depends on 3)

The method of claim 3, wherein the first and second LFEs are different type LFEs.

### Claim 6 (depends on 5)

The method of claim 5, wherein the first LFE is a logical switch, while the second LFE is a logical router.

### Claim 7 (depends on 1)

The method of claim 1, wherein the controller assignment list associates each controller with more than one index value.

### Claim 8 (depends on 1)

The method of claim 1, wherein when a particular index value is derivable from first and second identifiers of first and second LFEs of the same type, a controller that is associated with the particular index value manages both the first and second LFEs.

### Claim 9 (depends on 1)

The method of claim 1, wherein when a particular index value is derivable from first and second identifiers of first and second LFEs of different types, a controller that is associated with the particular index value manages the first and second LFEs.

### Claim 10 (depends on 1)

The method of claim 1, wherein the index values are derivable from the LFE identifiers by using a hash operation that comprises a modulo operation, wherein the common range of index values is associated with a range of remainder values of the modulo operation.

### Claim 11

For a network control system that includes a plurality of host computers that execute a plurality of physical forwarding elements (PFEs) that collectively implement a plurality of logical forwarding elements (LFEs), a method of managing the LFEs comprising:
identifying a plurality of controllers to manage the LFEs;
generating a controller assignment list that associates each controller with an index value that is derivable from identifiers of the LFEs, said generating comprising assigning the index values to the controllers by using a load-balancing process for distributing the LFE management load across the controllers;
distributing the controller assignment list so that the list is received by the hosts, said hosts using the list to identify the controllers to contact to obtain data necessary for effectuating a subset of the LFE operations.

### Claim 12 (depends on 11)

The method of claim 11, wherein the load-balancing process distributes the responsibility for the LFEs based on a set of criteria that includes assigning substantially the same number of LFEs to each controller.

### Claim 13 (depends on 11)

The method of claim 11, wherein the load-balancing process distributes the responsibility for the LFEs based on a set of criteria that includes assigning substantially the same number of LFEs of the same type to each controller.

### Claim 14 (depends on 11)

The method of claim 11 further comprising:
detecting that one of the controllers has failed;
distributing the LFEs assigned to the failed controller to the remaining controllers by distributing the index values assigned to the failed controller to the remaining controllers.

### Claim 15 (depends on 14)

The method of claim 14, wherein distributing the index values comprises using the load-balancing process to distribute the index values that were previously assigned to the failed controller to the remaining controllers.

### Claim 16 (depends on 11)

The method of claim 11 further comprising:
detecting that a new controller has been added;
assigning a subset of LFEs that were previously assigned to other controllers to the new controller by re-assigning a subset of index values that were previously assigned to the other controllers to the new controller.

### Claim 17 (depends on 16)

The method of claim 16, wherein distributing the index values comprises using the load-balancing process to distribute an equal-sized or a substantially equal sized portion of index values that were previously assigned to each of the other controllers to the new controller.

### Claim 18 (depends on 11)

The method of claim 11, wherein distributing the controller assignment list comprises directly sending the controller assignment list to the host computers.

### Claim 19 (depends on 11)

The method of claim 11, wherein distributing the controller assignment list comprises sending the controller assignment list to the other controllers, wherein each controller provides the controller assignment list to a different set of host computers.

### Claim 20 (depends on 11)

The method of claim 11, wherein the identifying, generating and distributing operations are performed by one controller that is designated as a master controller for generating the controller assignment list.

### Claim 21 (depends on 11)

The method of claim 11 further comprising:
generating an index value based on the identifier of each LFE; and
using the generated index value to specify a controller on the controller assignment list.

### Claim 22 (depends on 11)

The method of claim 11 further comprising:
specifying a range of index values that corresponds to a range of remainder values for a modulo operation of a hash operation, wherein the hash operation is used by the host computers to generate the index values from the LFE identifiers.

### Claim 23

For a network control system that includes a plurality of host computers that execute a plurality of physical forwarding elements (PFEs) that collectively implement a plurality of logical forwarding elements (LFEs), a method of managing the LFEs comprising:
identifying a plurality of controllers to manage the LFEs;
gathering statistics regarding operations of the host computers relating to the LFEs;
based on the gathered statistics, assigning at least a subset of the LFEs to a subset of controllers;
receiving a request from a particular host computer to connect to a controller to receive data for processing packets associated with a particular LFE;
supplying the identity of a particular controller to the particular host computer, wherein the particular host computer uses the supplied identity to establish a control-channel connection with the particular controller, and uses the connection to process a subset of the operations associated with the particular LFE.

### Claim 24 (depends on 23)

The method of claim 23, wherein the particular host computer stores information about the connection in a connection list and then uses the connection list to subsequently identify the particular controller when the host computer needs a controller to process a packet associated with the particular LFE.
