# US 11,907,742 B2 · Software-defined network orchestration in a virtualized computer system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,907,742 B2 - Document ID: US11907742B2 - Status: Granted - Kind: B2 - Application number: 16/838,573 - Filed: 2020-04-02 - Published: 2021-10-07 - Granted: 2024-02-20 - Earliest claimed: 2020-04-02 - Assignees: VMware, Inc. - Applicants: VMware, Inc. - Inventors: Brian Masao Oki; Venkata N. V. Mukkavilli; Swapnil Dattatraya Tiwari; Vasantha K. Tammana - Web page: https://patentgrep.com/patent/US11907742B2 ## CPC - H04L 41/0806: for initial configuration or provisioning, e.g. plug-and-play - G06F 9/45558: {Hypervisor-specific management and integration aspects} - G06F 9/5077: {Logical partitioning of resources; Management or configuration of virtualized resources (specific details on emulation or internal functioning of virtual machines G06F9/455)} - G06F 9/547: {Remote procedure calls [RPC]; Web services} - H04L 12/4679: {Arrangements for the registration or de-registration of VLAN attribute values, e.g. VLAN identifiers, port VLAN membership} - H04L 12/66: Arrangements for connecting between networks having differing types of switching systems, e.g. gateways - H04L 41/0266: using meta-data, objects or commands for formatting management information, e.g. using eXtensible markup language [XML] - H04L 41/0893: Assignment of logical groups to network elements - H04L 41/0895: Configuration of virtualised networks or elements, e.g. virtualised network function or OpenFlow elements - H04L 41/122: of virtualised topologies, e.g. software-defined networks [SDN] or network function virtualisation [NFV] - H04L 41/20: {Network management software packages} - H04L 47/781: {Centralised allocation of resources} - H04L 47/803: {Application aware} - H04L 47/825: {Involving tunnels, e.g. MPLS} - H04L 47/828: {Allocation of resources per group of connections, e.g. per group of users} - H04L 61/5007: Internet protocol [IP] addresses - G06F 2009/45562: {Creating, deleting, cloning virtual machine instances} - G06F 2009/4557: {Distribution of virtual machine instances; Migration and load balancing} - G06F 2009/45595: {Network integration; Enabling network access in virtual machine instances} - H04L 12/4641: {Virtual LANs, VLANs, e.g. virtual private networks [VPN] (LAN interconnection over a bridge based backbone H04L12/462; encapsulation techniques H04L12/4633; routing of packets H04L45/00; packet switches H04L49/00; virtual private networks for security H04L63/0272)} - H04L 41/0843: {based on generic templates} - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract An example method of orchestrating a software-defined (SD) network layer of a virtualized computing system is described, the virtualized computing system including a host cluster, a virtualization management server, and a network management server each connected to a physical network, the host cluster having hosts and a virtualization layer executing on hardware platforms of the hosts. The method includes receiving, at the virtualization management server, a declarative specification describing a proposed state of an SD network for the host cluster, deploying, by the virtualization management server, virtualized infrastructure components in the host cluster in response to the proposed state in the declarative specification, and deploying, by the virtualization management server in cooperation with the network management server, logical network services supported by the virtualized infrastructure components in response to the proposed state in the declarative specification. ## Claims ### Claim 1 A method of orchestrating a software-defined (SD) network layer of a virtualized computing system, the virtualized computing system including a host cluster, a virtualization management server, and a network management server each connected to a physical network, the network management server being deployed in a separate physical or virtual server from the virtualization management server, the host cluster having hosts and a virtualization layer executing on hardware platforms of the hosts, the method comprising: receiving, at the virtualization management server, a declarative specification describing a proposed state of an SD network for the host cluster; deploying, by a network management service executing in the virtualization management server, virtualized infrastructure components in the host cluster needed to implement the SD network described in the declarative specification, the virtualized infrastructure including a resource pool, at least one virtual machine (VM) deployed in the resource pool, and a distributed switch deployed in the virtualization layer; and deploying, by the network management service in cooperation with the network management server, logical network services of the SD network on the virtualized infrastructure components, the logical network services including a logical switch deployed on the distributed switch and edge services deployed in the at least one VM. ### Claim 2 (depends on 1) The method of claim 1, wherein the virtualized infrastructure components include port groups on the distributed switch, the port groups implementing networks defined in the declarative specification. ### Claim 3 (depends on 2) The method of claim 2, wherein the declarative specification describes virtual local area network (VLAN) identifiers for the networks, and wherein each of the port groups is associated with a respective one of the VLAN identifiers. ### Claim 4 (depends on 2) The method of claim 2, wherein each of the at least one virtual machine connected to at least one of the port groups on the distributed switch, each of the at least one virtual machine having at least one virtual network interface having an Internet Protocol (IP) address defined in the declarative specification. ### Claim 5 (depends on 4) The method of claim 4, wherein the logical network services include gateway executing in a first VM of the at least one VM, wherein the networks defined in the declarative specification include a management network, an overlay network, and an uplink network, wherein the port groups include a first port group implementing the management network, a second port group implementing the overlay network, and a third port group implementing the uplink network, and wherein the gateway is connected to each of first, second, and third port groups. ### Claim 6 (depends on 4) The method of claim 4, wherein the logical network services include a tier-0 gateway executing in a first VM of the at least one VM, a tier-1 gateway implemented as a distributed router in the distributed switch, the tier-1 gateway connected to the tier-0 gateway, and at least one logical switch, implemented in the distributed switch, connected to the tier-1 gateway. ### Claim 7 (depends on 1) The method of claim 1, wherein the network management service orchestrates the deployment of the virtualized infrastructure components and the logical network services, the network management service performing the step of deploying the logical network services by invoking an application programming interface (API) of the network management server. ### Claim 8 (depends on 1) The method of claim 1, wherein the virtualization management server includes an orchestration service, wherein the orchestration service is configured to receive the declarative specification and cooperate with the network management service to perform the steps of deploying the virtualized infrastructure components and deploying the logical network services, the network management service invoking an application programming interface (API) of the network management server. ### Claim 9 (depends on 8) The method of claim 8, wherein the virtualization management server includes a supervisor cluster service configured to deploy an orchestration control plane integrated with the virtualization layer, wherein the supervisor cluster service provides the declarative specification to the orchestration service. ### Claim 10 (depends on 1) The method of claim 1, wherein the virtualization management server includes a supervisor cluster service configured to deploy an orchestration control plane integrated with the virtualization layer, wherein the supervisor cluster service provides the declarative specification to the network management service, and wherein the network management service orchestrates the deployment of the virtualized infrastructure components and the logical network services, the network management service performing the step of deploying the logical network services by invoking an application programming interface (API) of the network management server. ### Claim 11 A non-transitory computer readable medium comprising instructions to be executed in a computing device to cause the computing device to carry out a method of orchestrating a software-defined (SD) network layer of a virtualized computing system, the virtualized computing system including a host cluster, a virtualization management server, and a network management server each connected to a physical network, the network management server being deployed in a separate physical or virtual server from the virtualization management server, the host cluster having hosts and a virtualization layer executing on hardware platforms of the hosts, the method comprising: receiving, at the virtualization management server, a declarative specification describing a proposed state of an SD network for the host cluster; deploying, by a network management service executing in the virtualization management server, virtualized infrastructure components in the host cluster needed to implement the SD network described in the declarative specification, the virtualized infrastructure including a resource pool, at least one virtual machine (VM) deployed in the resource pool, and a distributed switch deployed in the virtualization layer; and deploying, by the network management service in cooperation with the network management server, logical network services of the SD network on the virtualized infrastructure components, the logical network services including a logical switch deployed on the distributed switch and edge services deployed in the at least one VM. ### Claim 12 (depends on 11) The non-transitory computer readable medium of claim 11, wherein: the virtualized infrastructure components include port groups on the distributed switch, the port groups implementing networks defined in the declarative specification; and wherein each of the at least one virtual machine connected to at least one of the port groups on the distributed switch, each of the at least one virtual machine having at least one virtual network interface having an Internet Protocol (IP) address defined in the declarative specification. ### Claim 13 (depends on 11) The non-transitory computer readable medium of claim 11, wherein the network management service orchestrates the deployment of the virtualized infrastructure components and the logical network services, the network management service performing the step of deploying the logical network services by invoking an application programming interface (API) of the network management server. ### Claim 14 (depends on 11) The non-transitory computer readable medium of claim 11, wherein the virtualization management server includes an orchestration service, wherein the orchestration service is configured to receive the declarative specification and cooperate with the network management service to perform the steps of deploying the virtualized infrastructure components and deploying the logical network services, the network management service invoking an application programming interface (API) of the network management server. ### Claim 15 (depends on 14) The non-transitory computer readable medium of claim 14, wherein the virtualization management server includes a supervisor cluster service configured to deploy an orchestration control plane integrated with the virtualization layer, wherein the supervisor cluster service provides the declarative specification to the orchestration service. ### Claim 16 (depends on 11) The non-transitory computer readable medium of claim 11, wherein the virtualization management server includes a supervisor cluster service configured to deploy an orchestration control plane integrated with the virtualization layer, wherein the supervisor cluster service provides the declarative specification to the network management service, and wherein the network management service orchestrates the deployment of the virtualized infrastructure components and the logical network services, the network management service performing the step of deploying the logical network services by invoking an application programming interface (API) of the network management server. ### Claim 17 A virtualized computing system, comprising: a host cluster, a virtualization management server, and a network management server each connected to a physical network, the network management server being deployed in a separate physical or virtual server from the virtualization management server; the host cluster including hosts and a virtualization layer executing on hardware platforms of the hosts; the network management server configured to manage an SD network for the host cluster; and the virtualization management server configured to: receive a declarative specification describing a proposed state of an SD network for the host cluster; deploying, by a network management service executing in the virtualization management server, virtualized infrastructure components in the host cluster needed to implement the SD network described in the declarative specification, the virtualized infrastructure including a resource pool, at least one virtual machine (VM) deployed in the resource pool, and a distributed switch deployed in the virtualization layer; and deploying, by the network management service in cooperation with the network management server, logical network services of the SD network on the virtualized infrastructure components, the logical network services including a logical switch deployed on the distributed switch and edge services deployed in the at least one VM. ### Claim 18 (depends on 17) The virtualized computing system of claim 17, wherein: the virtualized infrastructure components include port groups on the distributed switch, the port groups implementing networks defined in the declarative specification; and wherein each of the at least one virtual machine connected to at least one of the port groups on the distributed switch, each of the at least one virtual machine having at least one virtual network interface having an Internet Protocol (IP) address defined in the declarative specification. ### Claim 19 (depends on 18) The virtualized computing system of claim 18, wherein the logical network services include a gateway executing in a first VM of the at least one VM, wherein the networks defined in the declarative specification include a management network, an overlay network, and an uplink network, wherein the port groups include a first port group implementing the management network, an second port group implementing the overlay network, and a third port group implementing the uplink network, and wherein the gateway is connected to each of the first, the second, and the third port groups. ### Claim 20 (depends on 18) The virtualized computing system of claim 18, wherein the logical network services include a tier-0 gateway executing in a first VM of the at least one VM, a tier-1 gateway implemented as a distributed router in the distributed switch, the tier-1 gateway connected to the tier-0 gateway, and at least one logical switch, implemented in the distributed switch, connected to the tier-1 gateway. ### Claim 21 (depends on 17) The virtualized computing system of claim 17, wherein a management function of the SD network is divided between the virtualization management server and the network management server, wherein a control function of the SD network is exclusively performed by the network management server, and wherein a data function of the SD network is performed by the virtualization layer.