# US 12,010,172 B2 · Host device with multi-path layer configured for IO control using detected storage port resource availability

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,010,172 B2
- Document ID: US12010172B2
- Status: Granted
- Kind: B2
- Application number: 16/587,536
- Filed: 2019-09-30
- Published: 2021-04-01
- Granted: 2024-06-11
- Earliest claimed: 2019-09-30
- Assignees: EMC IP Holding Company LLC
- Applicants: EMC IP Holding Company LLC
- Inventors: Amit Pundalik Anchi; Rimpesh Patel
- Web page: https://patentgrep.com/patent/US12010172B2

## CPC

- H04L 67/1097: for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
- G06F 13/102: {where the program performs an interfacing function, e.g. device driver (G06F13/105 takes precedence; contention policies within device drivers G06F9/4881; scheduling within device drivers G06F9/52)}
- H04L 41/0246: Exchanging or transporting network management information using the Internet; Embedding network management web servers in network elements; Web-services-based protocols
- H04L 41/20: {Network management software packages}
- H04L 41/344: Out-of-band transfers
- H04L 43/0805: by checking availability
- H04L 67/141: Setup of application sessions (admission control or resource allocation in data switching networks H04L47/70)
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A host device is configured to communicate over a network with a storage system comprising a plurality of storage devices. The host device comprises a multi-path input-output (MPIO) driver configured to control delivery of input-output (IO) operations from the host device to ports of the storage system over selected ones of a plurality of paths through the network. The MPIO driver of the host device is further configured, for each of at least a subset of the ports, to obtain from the storage system information characterizing a current number of host devices connected to the port and a current available capacity measure of the port, and to limit an amount of the IO operations that are delivered over one or more of the paths to the port based at least in part on the obtained information.

## Claims

### Claim 1

An apparatus comprising:
a host device configured to communicate over a network with a storage system comprising a plurality of storage devices;
the host device comprising:
a multi-path input-output driver configured to control delivery of input-output operations from the host device to ports of the storage system over selected ones of a plurality of paths through the network;
wherein the multi-path input-output driver is further configured, for each of at least a subset of the ports of the storage system:
to obtain from the storage system information characterizing a current number of host devices connected to the port of the storage system and a current available capacity measure of the port of the storage system; and
to limit an amount of the input-output operations that are delivered over one or more of the paths to the port of the storage system based at least in part on the obtained information;
wherein the obtained information characterizing a current number of host devices connected to the port of the storage system and a current available capacity measure of the port of the storage system identifies the current number of host devices connected to the port of the storage system;
wherein obtaining from the storage system information characterizing a current number of host devices connected to the port and a current available capacity measure of the port comprises obtaining for a given one of a plurality of time periods the current number of host devices and the current available capacity measure for the given time period from a per-port availability data structure maintained by the storage system over the plurality of time periods; and
wherein the information obtained from the storage system characterizing the current number of host devices connected to the port and the current available capacity measure of the port is stored in a data structure of a kernel-space portion of the multi-path input-output driver.

### Claim 2 (depends on 1)

The apparatus of claim 1 further comprising one or more additional host devices each configured to communicate over the network with the storage system and wherein each additional host device comprises a multi-path input-output driver configured to control delivery of input-output operations from that host device to ports of the storage system over selected ones of a plurality of paths through the network.

### Claim 3 (depends on 1)

The apparatus of claim 1 wherein the paths are associated with respective initiator-target pairs with each of a plurality of initiators of the initiator-target pairs comprising a corresponding host bus adaptor of the host device and each of a plurality of targets of the initiator-target pairs comprising a corresponding one of the ports of the storage system.

### Claim 4 (depends on 1)

The apparatus of claim 1 wherein the current available capacity measure of the port comprises additional input-output operations per second supportable by the port beyond actual input-output operations per second already being supported by the port.

### Claim 5 (depends on 1)

The apparatus of claim 1 wherein the current available capacity measure of the port is dynamically determined by the storage system based at least in part on actual processing of input-output operations arriving at that port.

### Claim 6 (depends on 1)

The apparatus of claim 1 wherein the information characterizing a current number of host devices connected to the port and a current available capacity measure of the port is obtained by the host device from the storage system utilizing at least one of:
an in-band communication mechanism in which one or more commands in a designated storage protocol are sent from the host device to the storage system; and
an out-of-band communication mechanism in which host management software of the host device communicates with storage system management software of the storage system over an Internet Protocol (IP) network connection.

### Claim 7 (depends on 1)

The apparatus of claim 1 wherein obtaining from the storage system information characterizing a current number of host devices connected to the port and a current available capacity measure of the port comprises:
establishing a first time period; and
repeating the obtaining in each of a plurality of successive instances of the first time period so as to repeatedly obtain from the storage system for each such instance the information characterizing a current number of host devices connected to the port and a current available capacity measure of the port.

### Claim 8 (depends on 1)

The apparatus of claim 1 wherein limiting an amount of the input-output operations that are delivered over one or more of the paths to the port based at least in part on the obtained information comprises:
computing effective permissible input-output operations per second for the port based at least in part on the current number of host devices connected to the port and the current available capacity measure of the port; and
limiting the amount of the input-output operations that are delivered over one or more of the paths to the port based at least in part on the effective permissible input-output operations per second for the port.

### Claim 9

An apparatus comprising:
a host device configured to communicate over a network with a storage system comprising a plurality of storage devices;
the host device comprising:
a multi-path input-output driver configured to control delivery of input-output operations from the host device to ports of the storage system over selected ones of a plurality of paths through the network;
wherein the multi-path input-output driver is further configured, for each of at least a subset of the ports of the storage system:
to obtain from the storage system information characterizing a current number of host devices connected to the port of the storage system and a current available capacity measure of the port of the storage system; and
to limit an amount of the input-output operations that are delivered over one or more of the paths to the port of the storage system based at least in part on the obtained information;
wherein the obtained information characterizing a current number of host devices connected to the port of the storage system and a current available capacity measure of the port of the storage system identifies the current number of host devices connected to the port of the storage system;
wherein obtaining from the storage system information characterizing a current number of host devices connected to the port and a current available capacity measure of the port comprises obtaining for a given one of a plurality of time periods the current number of host devices and the current available capacity measure for the given time period from a per-port availability data structure maintained by the storage system over the plurality of time periods;
wherein limiting an amount of the input-output operations that are delivered over one or more of the paths to the port based at least in part on the obtained information comprises:
computing effective permissible input-output operations per second for the port based at least in part on the current number of host devices connected to the port and the current available capacity measure of the port; and
limiting the amount of the input-output operations that are delivered over one or more of the paths to the port based at least in part on the effective permissible input-output operations per second for the port; and
wherein the effective permissible input-output operations per second for the port is computed as a function of a rate-limiting factor of the host device and a ratio of the current available capacity measure of the port and the current number of host devices connected to the port.

### Claim 10 (depends on 9)

The apparatus of claim 9 wherein different rate-limiting factors are utilized by different ones of a plurality of host devices configured to communicate over the network with the storage system in computing their respective effective permissible input-output operations per second for the port.

### Claim 11

An apparatus comprising:
a host device configured to communicate over a network with a storage system comprising a plurality of storage devices;
the host device comprising:
a multi-path input-output driver configured to control delivery of input-output operations from the host device to ports of the storage system over selected ones of a plurality of paths through the network;
wherein the multi-path input-output driver is further configured, for each of at least a subset of the ports of the storage system:
to obtain from the storage system information characterizing a current number of host devices connected to the port of the storage system and a current available capacity measure of the port of the storage system; and
to limit an amount of the input-output operations that are delivered over one or more of the paths to the port of the storage system based at least in part on the obtained information;
wherein the obtained information characterizing a current number of host devices connected to the port of the storage system and a current available capacity measure of the port of the storage system identifies the current number of host devices connected to the port of the storage system;
wherein obtaining from the storage system information characterizing a current number of host devices connected to the port and a current available capacity measure of the port comprises obtaining for a given one of a plurality of time periods the current number of host devices and the current available capacity measure for the given time period from a per-port availability data structure maintained by the storage system over the plurality of time periods;
wherein obtaining from the storage system information characterizing a current number of host devices connected to the port and a current available capacity measure of the port comprises:
establishing a first time period; and
repeating the obtaining in each of a plurality of successive instances of the first time period so as to repeatedly obtain from the storage system for each such instance the information characterizing a current number of host devices connected to the port and a current available capacity measure of the port; and
wherein limiting an amount of the input-output operations that are delivered over one or more of the paths to the port based at least in part on the obtained information comprises:
establishing a second time period having a duration less than that of the first time period;
computing actual input-output operations per second for the port for a given instance of the second time period;
comparing the actual input-output operations per second for the port to effective permissible input-output operations per second for the port computed using the obtained information for a corresponding instance of the first time period; and
limiting the amount of the input-output operations that are delivered over one or more of the paths to the port based at least in part on a result of the comparing.

### Claim 12 (depends on 11)

The apparatus of claim 11 wherein the first time period has a duration that is an integer multiple of the duration of the second time period.

### Claim 13 (depends on 11)

The apparatus of claim 11 wherein the duration of the second time period is reduced responsive to the actual input-output operations per second for the port reaching a specified threshold level of input-output operations per second.

### Claim 14

A method comprising:
configuring a multi-path input-output driver of a host device to control delivery of input-output operations from the host device to ports of a storage system over selected ones of a plurality of paths through a network;
the multi-path input-output driver, for each of at least a subset of the ports of the storage system:
obtaining from the storage system information characterizing a current number of host devices connected to the port of the storage system and a current available capacity measure of the port of the storage system; and
limiting an amount of the input-output operations that are delivered over one or more of the paths to the port of the storage system based at least in part on the obtained information;
wherein the obtained information characterizing a current number of host devices connected to the port of the storage system and a current available capacity measure of the port of the storage system identifies the current number of host devices connected to the port of the storage system;
wherein obtaining from the storage system information characterizing a current number of host devices connected to the port and a current available capacity measure of the port comprises obtaining for a given one of a plurality of time periods the current number of host devices and the current available capacity measure for the given time period from a per-port availability data structure maintained by the storage system over the plurality of time periods; and
wherein the information obtained from the storage system characterizing the current number of host devices connected to the port and the current available capacity measure of the port is stored in a data structure of a kernel-space portion of the multi-path input-output driver.

### Claim 15 (depends on 14)

The method of claim 14 wherein obtaining from the storage system information characterizing a current number of host devices connected to the port and a current available capacity measure of the port comprises:
establishing a first time period; and
repeating the obtaining in each of a plurality of successive instances of the first time period so as to repeatedly obtain from the storage system for each such instance the information characterizing a current number of host devices connected to the port and a current available capacity measure of the port.

### Claim 16 (depends on 14)

The method of claim 14 wherein limiting an amount of the input-output operations that are delivered over one or more of the paths to the port based at least in part on the obtained information comprises:
computing effective permissible input-output operations per second for the port based at least in part on the current number of host devices connected to the port and the current available capacity measure of the port; and
limiting the amount of the input-output operations that are delivered over one or more of the paths to the port based at least in part on the effective permissible input-output operations per second for the port.

### Claim 17

A computer program product comprising a non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code, when executed by a host device comprising a multi-path input-output driver, the host device being configured to communicate over a network with a storage system, causes the multi-path input-output driver:
to control delivery of input-output operations from the host device to ports of the storage system over selected ones of a plurality of paths through the network; and
for each of at least a subset of the ports of the storage system:
to obtain from the storage system information characterizing a current number of host devices connected to the port of the storage system and a current available capacity measure of the port of the storage system; and
to limit an amount of the input-output operations that are delivered over one or more of the paths to the port of the storage system based at least in part on the obtained information;
wherein the obtained information characterizing a current number of host devices connected to the port of the storage system and a current available capacity measure of the port of the storage system identifies the current number of host devices connected to the port of the storage system;
wherein obtaining from the storage system information characterizing a current number of host devices connected to the port and a current available capacity measure of the port comprises obtaining for a given one of a plurality of time periods the current number of host devices and the current available capacity measure for the given time period from a per-port availability data structure maintained by the storage system over the plurality of time periods; and
wherein the information obtained from the storage system characterizing the current number of host devices connected to the port and the current available capacity measure of the port is stored in a data structure of a kernel-space portion of the multi-path input-output driver.

### Claim 18 (depends on 17)

The computer program product of claim 17 wherein obtaining from the storage system information characterizing a current number of host devices connected to the port and a current available capacity measure of the port comprises:
establishing a first time period; and
repeating the obtaining in each of a plurality of successive instances of the first time period so as to repeatedly obtain from the storage system for each such instance the information characterizing a current number of host devices connected to the port and a current available capacity measure of the port.

### Claim 19 (depends on 17)

The computer program product of claim 17 wherein limiting an amount of the input-output operations that are delivered over one or more of the paths to the port based at least in part on the obtained information comprises:
computing effective permissible input-output operations per second for the port based at least in part on the current number of host devices connected to the port and the current available capacity measure of the port; and
limiting the amount of the input-output operations that are delivered over one or more of the paths to the port based at least in part on the effective permissible input-output operations per second for the port.

### Claim 20 (depends on 19)

The computer program product of claim 19 wherein the effective permissible input-output operations per second for the port is computed as a function of a rate-limiting factor of the host device and a ratio of the current available capacity measure of the port and the current number of host devices connected to the port.
