# US 9,882,969 B2 · Methods and apparatus to configure virtual resource managers for use in virtual server rack deployments for virtual computing environments

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 9,882,969 B2
- Document ID: US9882969B2
- Status: Granted
- Kind: B2
- Application number: 14/796,803
- Filed: 2015-07-10
- Published: 2016-01-14
- Granted: 2018-01-30
- Earliest claimed: 2014-07-11
- Assignees: VMware, Inc.
- Applicants: VMware, Inc.
- Inventors: Thirumalesh Reddy; Raj Yavatkar; Sreekantha Indireddy; Shreekanth Ankala
- Web page: https://patentgrep.com/patent/US9882969B2

## CPC

- G06F 9/45558: {Hypervisor-specific management and integration aspects}
- G06F 8/61: Installation
- G06F 9/4408: {Boot device selection}
- G06F 9/4416: {Network booting; Remote initial program loading [RIPL]}
- G06F 9/5077: {Logical partitioning of resources; Management or configuration of virtualized resources (specific details on emulation or internal functioning of virtual machines G06F9/455)}
- H04L 12/4675: {Dynamic sharing of VLAN information amongst network nodes (configuration of the network or of network elements H04L41/08)}
- H04L 41/048: {mobile agents}
- H04L 41/0806: for initial configuration or provisioning, e.g. plug-and-play
- H04L 41/0895: Configuration of virtualised networks or elements, e.g. virtualised network function or OpenFlow elements
- H04L 41/0897: by horizontal or vertical scaling of resources, or by migrating entities, e.g. virtual resources or entities
- H04L 41/40: using virtualisation of network functions or resources, e.g. SDN or NFV entities
- H04L 41/5009: Determining service level performance parameters or violations of service level contracts, e.g. violations of agreed response time or mean time between failures [MTBF]
- H04L 41/5025: by proactively reacting to service quality change, e.g. by reconfiguration after service quality degradation or upgrade
- H04L 41/5054: Automatic deployment of services triggered by the service manager, e.g. service implementation by automatic configuration of network components
- H04L 41/5096: {wherein the managed service relates to distributed or central networked applications}
- H04L 43/065: related to network devices
- H04L 43/16: Threshold monitoring
- H04L 47/70: Admission control; Resource allocation
- H04L 47/822: {Collecting or measuring resource availability data}
- H04L 67/10: in which an application is distributed across nodes in the network (software deployment G06F8/60; multiprogramming arrangements G06F9/46)
- H04L 67/131: Protocols for games, networked simulations or virtual reality
- G06F 2009/45595: {Network integration; Enabling network access in virtual machine instances}
- H04L 41/0266: using meta-data, objects or commands for formatting management information, e.g. using eXtensible markup language [XML]
- H04L 41/0681: Configuration of triggering conditions
- H04L 41/082: the condition being updates or upgrades of network functionality
- H04L 43/0817: by checking functioning
- H04L 43/0852: Delays
- H04L 43/0876: Network utilisation, e.g. volume of load or congestion level
- H04L 43/10: Active monitoring, e.g. heartbeat, ping or trace-route
- H04L 43/20: the monitoring system or the monitored elements being virtualised, abstracted or software-defined entities, e.g. SDN or NFV
- H04L 43/55: Testing of service level quality, e.g. simulating service usage
- H04L 61/5014: using dynamic host configuration protocol [DHCP] or bootstrap protocol [BOOTP]
- H04L 61/5076: Update or notification mechanisms, e.g. DynDNS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

An example method includes configuring a network to facilitate communications between a first virtual resource manager (VRM) in a first physical rack and a second VRM in a second physical rack. The first VRM manages first physical resources in the first physical rack. The second VRM manages second physical resources in the second physical rack. A primary election process determines one of the first or second VRMs to operate as a primary VRM for a virtual server rack. The virtual server rack is based on the first physical resources and the second physical resources. When the first VRM is the elected primary VRM, the first VRM is registered as the primary VRM. When the first VRM is not the elected primary VRM, the first VRM is registered as a secondary VRM, and a watch object is created to monitor for a failure or unavailability of the primary VRM.

## Claims

### Claim 1

A method to operate a virtual resource manager, the method comprising:
in response to a power-on event, configuring a network to facilitate communications between a first virtual resource manager in a first physical rack and a second virtual resource manager in a second physical rack, the first virtual resource manager to present a first logical resource provisioning of first physical hardware resources in the first physical rack to first virtualization components in the first physical rack, and the second virtual resource manager to present a second logical resource provisioning of second physical hardware resources in the second physical rack to second virtualization components in the second physical rack, the first virtual resource manager to translate the first logical resource provisioning to an allocation of the first physical hardware resources for use by a virtual server rack and the second virtual resource manager to translate the second logical resource provisioning to an allocation of the second physical hardware resources for use by the virtual server rack, the first virtualization components and the second virtualization components to execute workloads using the first logical resource provisioning and the second logical resource provisioning supported by the first physical hardware resources and second physical hardware resources;
performing a primary election process to determine one of the first or second virtual resource managers to operate as a primary virtual resource manager for the virtual server rack, the virtual server rack being an abstraction of the allocation of the first physical hardware resources of the first physical rack and the allocation of the second physical hardware resources of the second physical rack;
when the first virtual resource manager is the elected primary virtual resource manager, registering the first virtual resource manager as the primary virtual resource manager; and
when the first virtual resource manager is not the elected primary virtual resource manager:
registering the first virtual resource manager as a secondary virtual resource manager, and
creating a watch object to monitor for a failure or unavailability of the primary virtual resource manager.

### Claim 2 (depends on 1)

A method as defined in claim 1, wherein when the first virtual resource manager is the primary virtual resource manager and the second virtual resource manager is the secondary virtual resource manager, the primary virtual resource manager manages the first physical hardware resources of the first physical rack and delegates instructions to the secondary virtual resource manager to cause the secondary virtual resource manager to manage the second physical hardware resources of the second physical rack.

### Claim 3 (depends on 1)

A method as defined in claim 1, further comprising when the primary virtual resource manager fails, performing the primary election process to determine another one of the first or second virtual resource managers to operate as the primary virtual resource manager.

### Claim 4 (depends on 1)

A method as defined in claim 1, further comprising, in response to the power-on event, sending an election initialization message from the first virtual resource manager to the second virtual resource manager to begin the primary election process.

### Claim 5 (depends on 4)

A method as defined in claim 4, further comprising:
receiving a response message at the first virtual resource manager from the second virtual resource manager, the response message including configuration information of the second virtual resource manager; and
performing the primary election process at the first virtual resource manager to elect the primary virtual resource manager based on the configuration information.

### Claim 6 (depends on 1)

A method as defined in claim 1, wherein the registering of the first virtual resource manager as the primary virtual resource manager includes registering the first virtual resource manager as the primary virtual resource manager in a zookeeper of the first virtual resource manager.

### Claim 7 (depends on 6)

A method as defined in claim 6, wherein the zookeeper is a cluster configuration manager to monitor and synchronize at least one of data or tasks across the first and second physical racks, the at least one of data or tasks pertaining to management processes of the first and second virtual resource managers to manage the first and second physical hardware resources.

### Claim 8

An apparatus comprising:
a network virtualizer to, in response to a power-on event, configure a network to facilitate communications between a first virtual resource manager in a first physical rack and a second virtual resource manager in a second physical rack, the first virtual resource manager to present a first logical resource provisioning of first physical hardware resources in the first physical rack to first virtualization components in the first physical rack, and the second virtual resource manager to present a second logical resource provisioning of second physical hardware resources in the second physical rack to second virtualization components in the second physical rack, the first virtual resource manager to translate the first logical resource provisioning to an allocation of the first physical hardware resources for use by a virtual server rack and the second virtual resource manager to translate the second logical resource provisioning to an allocation of the second physical hardware resources for use by the virtual server rack, respectively, the first virtualization components and the second virtualization components to execute workloads using the first logical resource provisioning and the second logical resource provisioning supported by the first physical hardware resources and second physical hardware resources;
an election manager to perform a primary election process to determine one of the first or second virtual resource managers to operate as a primary virtual resource manager for the virtual server rack, the virtual server rack being an abstraction of the allocation of the first physical hardware resources of the first physical rack and the allocation of the second physical hardware resources of the second physical rack;
a processor to:
when the first virtual resource manager is the elected primary virtual resource manager, register the first virtual resource manager as the primary virtual resource manager; and
when the first virtual resource manager is not the elected primary virtual resource manager:
register the first virtual resource manager as a secondary virtual resource manager, and
create a watch object to monitor for a failure or unavailability of the primary virtual resource manager.

### Claim 9 (depends on 8)

An apparatus as defined in claim 8, wherein, when the first virtual resource manager is the primary virtual resource manager and the second virtual resource manager is the secondary virtual resource manager, the primary virtual resource manager is to manage the first physical hardware resources of the first physical rack and is to delegate instructions to the secondary virtual resource manager to cause the secondary virtual resource manager to manage the second physical hardware resources of the second physical rack.

### Claim 10 (depends on 8)

An apparatus as defined in claim 8, wherein when the primary virtual resource manager fails, the election manager is to perform the primary election process to determine another one of the first or second virtual resource managers to operate as the primary virtual resource manager.

### Claim 11 (depends on 8)

An apparatus as defined in claim 8, wherein, in response to the power-on event, the election manager is to send an election initialization message from the first virtual resource manager to the second virtual resource manager to begin the primary election process.

### Claim 12 (depends on 11)

An apparatus as defined in claim 11, wherein the election manager is further to:
perform the primary election process at the first virtual resource manager to elect the primary virtual resource manager based on configuration information of the second virtual resource manager, the configuration information received from the second virtual resource manager.

### Claim 13 (depends on 8)

An apparatus as defined in claim 8, wherein the processor is to register the first virtual resource manager as the primary virtual resource manager by registering the first virtual resource manager as the primary virtual resource manager in a zookeeper of the first virtual resource manager.

### Claim 14 (depends on 13)

An apparatus as defined in claim 13, wherein the zookeeper is to monitor and synchronize at least one of data or tasks across the first and second physical racks, the at least one of data or tasks pertaining to management processes of the first and second virtual resource managers to manage the first and second physical hardware resources.

### Claim 15

A non-transitory computer readable medium comprising instructions that, when executed, cause a processor to at least:
in response to a power-on event, configure a network to facilitate communications between a first virtual resource manager in a first physical rack and a second virtual resource manager in a second physical rack, the first virtual resource manager to present a first logical resource provisioning of first physical hardware resources in the first physical rack to first virtualization components in the first physical rack, and the second virtual resource manager to present a second logical resource provisioning of second physical hardware resources in the second physical rack to second virtualization components in the second physical rack, the first virtual resource manager to translate the first logical resource provisioning to an allocation of the first physical hardware resources for use by a virtual server rack and the second virtual resource manager to translate the second logical resource provisioning to an allocation of the second physical hardware resources for use by the virtual server rack, the first virtualization components and the second virtualization components to execute workloads using the first logical resource provisioning and the second logical resource provisioning supported by the first physical hardware resources and second physical hardware resources;
perform a primary election process to determine one of the first or second virtual resource managers to operate as a primary virtual resource manager for the virtual server rack, the virtual server rack being an abstraction of the allocation of the first physical hardware resources of the first physical rack and the allocation of the second physical hardware resources of the second physical rack;
when the first virtual resource manager is the elected primary virtual resource manager, register the first virtual resource manager as the primary virtual resource manager; and
when the first virtual resource manager is not the elected primary virtual resource manager:
register the first virtual resource manager as a secondary virtual resource manager, and
create a watch object to monitor for a failure or unavailability of the primary virtual resource manager.

### Claim 16 (depends on 15)

A non-transitory computer readable medium as defined in claim 15, wherein when the first virtual resource manager is the primary virtual resource manager and the second virtual resource manager is the secondary virtual resource manager, the primary virtual resource manager manages the first physical hardware resources of the first physical rack and delegates instructions to the secondary virtual resource manager to cause the secondary virtual resource manager to manage the second physical hardware resources of the second physical rack.

### Claim 17 (depends on 15)

A non-transitory computer readable medium as defined in claim 15, wherein the instructions are further to cause the processor to, when the primary virtual resource manager fails, perform the primary election process to determine another one of the first or second virtual resource managers to operate as the primary virtual resource manager.

### Claim 18 (depends on 15)

A non-transitory computer readable medium as defined in claim 15, wherein the instructions are further to cause the processor to, in response to the power-on event, send an election initialization message from the first virtual resource manager to the second virtual resource manager to begin the primary election process.

### Claim 19 (depends on 18)

A non-transitory computer readable medium as defined in claim 18, wherein the instructions are further to:
receive a response message at the first virtual resource manager from the second virtual resource manager, the response message including configuration information of the second virtual resource manager; and
perform the primary election process at the first virtual resource manager to elect the primary virtual resource manager based on the configuration information.

### Claim 20 (depends on 15)

A non-transitory computer readable medium as defined in claim 15, wherein registering the first virtual resource manager as the primary virtual resource manager includes registering the first virtual resource manager as the primary virtual resource manager in a zookeeper of the first virtual resource manager.

### Claim 21 (depends on 20)

A non-transitory computer readable medium as defined in claim 20, wherein the zookeeper is a cluster configuration manager to monitor and synchronize at least one of data or tasks across the first and second physical racks, the at least one of data or tasks pertaining to management processes of the first and second virtual resource managers to manage the first and second physical hardware resources.