# US 2017/0286283 A1 · Data distribution method in storage system, distribution apparatus, and storage system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2017/0286283 A1
- Document ID: US20170286283A1
- Status: Published application
- Kind: A1
- Application number: 15/631,888
- Filed: 2017-06-23
- Published: 2017-10-05
- Earliest claimed: 2014-12-27
- Assignees: Huawei Technologies Co., Ltd.
- Applicants: Huawei Technologies Co.,Ltd.
- Inventors: Huifeng Xu; Wei Zha; Yadong Wu; Baifeng Yu; Yu Zhang; Changkun Ma
- Web page: https://patentgrep.com/patent/US20170286283A1

## CPC

- G06F 3/061: {Improving I/O performance}
- G06F 3/0604: {Improving or facilitating administration, e.g. storage management}
- G06F 3/0617: {in relation to availability}
- G06F 3/0658: {Controller construction arrangements}
- G06F 3/0683: {Plurality of storage devices}
- G11C 7/00: Arrangements for writing information into, or reading information out from, a digital store (G11C5/00 takes precedence; auxiliary circuits for stores using semiconductor devices G11C11/4063, G11C11/413)
- G06F 3/0601: {Interfaces specially adapted for storage systems}
- G06F 3/0602: {specifically adapted to achieve a particular effect}
- G06F 3/0631: {by allocating resources to storage systems}
- G06F 3/0635: {by changing the path, e.g. traffic rerouting, path reconfiguration}
- G06F 12/0868: Data transfer between cache memory and other subsystems, e.g. storage devices or host systems
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G11C: STATIC STORES (semiconductor memory devices H10B)

## Abstract

Embodiments of the present disclosure provide a storage system, including a distribution apparatus and a storage device; the storage device includes at least two storage controllers and multiple storage units, where each storage unit is configured with any one of the at least two storage controllers as a home storage controller; and the distribution apparatus includes a front-end interface and at least two back-end interfaces, where the front-end interface is configured to connect to a host device, and each back-end interface is connected to each storage controller in a one-to-one correspondence manner. According to the technical solutions provided in the present disclosure, the distribution apparatus parses an IO read/write instruction, so that the IO read/write instruction can be accurately sent to the home storage controller, which avoids forwarding the IO read/write instruction between the storage controllers, thereby improving IO processing efficiency of the storage system.

## Claims

### Claim 1

A storage system, comprising:
a storage device comprising at least two storage controllers and multiple storage units, with each storage unit configured with any one of the at least two storage controllers as a home storage controller;
a distribution apparatus comprising:
a front-end interface configured to connect to a host device; and
at least two back-end interfaces, wherein each back-end interface is configured to connect to each storage controller in a one-to-one correspondence manner; and
the distribution apparatus is configured to:
receive, using the front-end interface, an IO read/write instruction sent by the host device;
determine, according to an identifier of a to-be-read/written storage unit and carried in the IO read/write instruction, a home storage controller corresponding to the to-be-read/written storage unit; and
forward, using a back-end interface connected to the home storage controller corresponding to the to-be-read/written storage unit, the IO read/write instruction to the home storage controller corresponding to the to-be-read/written storage unit.

### Claim 2 (depends on 1)

The storage system according to claim 1, wherein the distribution apparatus is further configured to save route configuration information, and the route configuration information is used to record a correspondence between each storage unit and the home storage controller configured for each storage unit; and
the distribution apparatus is configured to search the route configuration information according to the identifier of the to-be-read/written storage unit and carried in the IO read/write instruction, to determine the home storage controller corresponding to the to-be-read/written storage unit.

### Claim 3 (depends on 2)

The storage system according to claim 2, wherein the distribution apparatus is further configured to receive information about a storage unit homed by each storage controller and initially configured in the system, and establish the route configuration information according to the received information about the storage unit homed by each storage controller and initially configured in the system.

### Claim 4 (depends on 3)

The storage system according to claim 3, wherein each storage controller is further configured to establish a communication link to the distribution apparatus during system initialization, and send, to the distribution apparatus using the communication link, the information about the storage unit homed by the storage controller and that is initially configured in the system.

### Claim 5 (depends on 3)

The storage system according to claim 3, wherein
the distribution apparatus is further configured to receive updated information about the storage unit homed by each storage controller, and update the route configuration information according to the received updated information about the storage unit homed by each storage controller.

### Claim 6 (depends on 1)

The storage system according to claim 1, wherein the home storage controller corresponding to the to-be-read/written storage unit is further configured to:
after receiving the IO read/write instruction, allocate a storage address from a cache or memory of the home storage controller for the IO read/write instruction; and
send the storage address to the distribution apparatus; and
the distribution apparatus is further configured to forward the received storage address to the host device.

### Claim 7 (depends on 6)

The storage system according to claim 6, wherein the IO read/write instruction is a write operation instruction, and the home storage controller corresponding to the to-be-read/written storage unit is further configured to receive data corresponding to the IO read/write instruction and written at the storage address by the host device in a direct memory access (DMA) manner.

### Claim 8 (depends on 6)

The storage system according to claim 6, wherein the IO read/write instruction is a read operation instruction, and the home storage controller corresponding to the to-be-read/written storage unit is further configured to receive a direct memory access (DMA) read request sent by the host device according to the storage address, and send, to the host device, data corresponding to the IO read/write instruction and at the storage address.

### Claim 9 (depends on 8)

The storage system according to claim 8, wherein the home storage controller corresponding to the to-be-read/written storage unit is configured to:
after receiving the read operation instruction, determine whether the data corresponding to the read operation instruction is saved in the cache or memory of the home storage controller; and if the data corresponding to the read operation instruction is saved in the cache or memory of the home storage controller, acquire the storage address of the data corresponding to the read operation instruction and in the cache or memory of the home storage controller; or if the data corresponding to the read operation instruction is not saved in the cache or memory of the home storage controller, read, from the to-be-read/written storage unit, the data corresponding to the read operation instruction, write the data into the cache or memory of the home storage controller, and use a written address as the storage address.

### Claim 10

A storage system, comprising:
a storage device, comprising:
at least two storage controllers; and
multiple storage units, wherein each storage unit is configured with any one of the at least two storage controllers as a home storage controller;
a distribution apparatus, comprising:
a processing unit;
a host interaction unit; and
a switch unit coupled to the processing unit, wherein the host interaction unit is connected to the processing unit using the switch unit;
the host interaction unit is provided with a front-end interface, wherein the front-end interface is configured to connect to a host device, and the switch unit is provided with at least two back-end interfaces, wherein each back-end interface is connected to each storage controller in a one-to-one correspondence manner;
the host interaction unit is configured to:
receive, using the front-end interface, an IO read/write instruction sent by the host device; and
forward the IO read/write instruction to the processing unit using the switch unit; and
the processing unit is configured to:
receive the IO read/write instruction forwarded by the host interaction unit;
determine, according to an identifier of a to-be-read/written storage unit and carried in the IO read/write instruction, a home storage controller corresponding to the to-be-read/written storage unit; and
forward, using a back-end interface connected to the home storage controller corresponding to the to-be-read/written storage unit and provided in the switch unit, the IO read/write instruction to the home storage controller corresponding to the to-be-read/written storage unit.

### Claim 11 (depends on 10)

The storage system according to claim 10, wherein a Peripheral Component Interconnect Express (PCIe) link is used for connection between the processing unit and the switch unit and between the host interaction unit and the switch unit, and the processing unit is configured as a PCIe root complex of the host interaction unit and each back-end interface;
the host interaction unit is configured to:
receive the IO read/write instruction sent by the host device; and
send the IO read/write instruction to the PCIe root complex using the PCIe link; and
the processing unit, as the PCIe root complex, is configured to receive the IO read/write instruction sent by the host interaction unit using the PCIe link.

### Claim 12 (depends on 10)

The storage system according to claim 10, wherein each back-end interface is connected, using a Peripheral Component Interconnect Express (PCIe) none transparent bridge, to each storage controller that is connected in a one-to-one correspondence manner;
the processing unit is configured to:
after receiving the IO read/write instruction, determine a back-end interface connected to the home storage controller corresponding to the to-be-read/written storage unit;
fill a destination address of the IO read/write instruction as an address of the back-end interface connected to the home storage controller corresponding to the to-be-read/written storage unit; and
send the IO read/write instruction using the PCIe link; and
the back-end interface connected to the home storage controller corresponding to the to-be-read/written storage unit is configured to:
receive the IO read/write instruction;
map the destination address of the IO read/write instruction to an address of the home storage controller corresponding to the to-be-read/written storage unit; and
send the IO read/write instruction using the PCIe none transparent bridge.

### Claim 13 (depends on 10)

The storage system according to claim 10, wherein the home storage controller corresponding to the to-be-read/written storage unit is further configured to:
after receiving the IO read/write instruction, allocate a storage address from a cache or memory of the home storage controller for the IO read/write instruction; and
send the storage address to the processing unit of the distribution apparatus; and
the processing unit is further configured to forward the received storage address to the host device so the host device operates, according to the storage address, data corresponding to the IO read/write instruction.

### Claim 14 (depends on 13)

The storage system according to claim 13, wherein the IO read/write instruction is a write operation instruction, and the home storage controller corresponding to the to-be-read/written storage unit is further configured to receive the data corresponding to the IO read/write instruction written at the storage address by the host device in a direct memory access (DMA) manner.

### Claim 15 (depends on 13)

The storage system according to claim 13, wherein the IO read/write instruction is a read operation instruction, and the home storage controller corresponding to the to-be-read/written storage unit is further configured to:
receive a direct memory access (DMA) read request sent by the host device according to the storage address; and
send, to the host device, the data corresponding to the IO read/write instruction and at the storage address.

### Claim 16 (depends on 15)

The storage system according to claim 15, wherein the home storage controller corresponding to the to-be-read/written storage unit is configured to:
after receiving the read operation instruction, determine whether the data corresponding to the read operation instruction is saved in the cache or memory of the home storage controller; and
if the data corresponding to the read operation instruction is saved in the cache or memory of the home storage controller, acquire the storage address of the data corresponding to the read operation instruction and in the cache or memory of the home storage controller; or if the data corresponding to the read operation instruction is not saved in the cache or memory of the home storage controller, read, from the to-be-read/written storage unit, the data corresponding to the read operation instruction, write the data into the cache or memory of the home storage controller, and use a written address as the storage address.

### Claim 17 (depends on 10)

The storage system according to claim 10, wherein the distribution apparatus is further configured to save route configuration information, and the route configuration information is used to record a correspondence between each storage unit and the home storage controller configured for each storage unit; and
the processing unit is configured to search the route configuration information according to the identifier of the to-be-read/written storage unit and carried in the TO read/write instruction, to determine the home storage controller corresponding to the to-be-read/written storage unit.

### Claim 18 (depends on 17)

The storage system according to claim 17, wherein the processing unit is further configured to:
receive information about a storage unit homed by each storage controller and initially configured in the system; and
establish the route configuration information according to the received information about the storage unit homed by each storage controller and initially configured in the system.

### Claim 19 (depends on 18)

The storage system according to claim 18, wherein each storage controller is further configured to:
establish a communication link to the distribution apparatus during system initialization; and
send, to the processing unit of the distribution apparatus using the communication link, the information about the storage unit homed by the storage controller and initially configured in the system.

### Claim 20 (depends on 18)

The storage system according to claim 18, wherein the processing unit is further configured to:
receive updated information about the storage unit homed by each storage controller; and
update the route configuration information according to the received updated information about the storage unit homed by each storage controller.

### Claim 21

A distribution apparatus, comprising:
a processing unit,
a host interaction unit, and
a switch unit coupled to the processing unit, wherein the host interaction unit is connected to the processing unit using the switch unit;
the host interaction unit is provided with a front-end interface, wherein the front-end interface is configured to connect to a host device; and the switch unit is provided with at least two back-end interfaces, and the at least two back-end interfaces are configured to connect to at least two storage controllers in a storage device, wherein each back-end interface is connected to each storage controller in a one-to-one correspondence manner, and each storage unit of the storage device is configured with any one of the at least two storage controllers as a home storage controller;
the host interaction unit is configured to:
receive, using the front-end interface, an IO read/write instruction sent by the host device; and
forward the IO read/write instruction to the processing unit using the switch unit; and
the processing unit is configured to:
receive the IO read/write instruction forwarded by the host interaction unit;
determine, according to an identifier of a to-be-read/written storage unit and carried in the IO read/write instruction, a home storage controller corresponding to the to-be-read/written storage unit; and
forward, using a back-end interface connected to the home storage controller corresponding to the to-be-read/written storage unit and provided in the switch unit, the IO read/write instruction to the home storage controller corresponding to the to-be-read/written storage unit.

### Claim 22 (depends on 21)

The distribution apparatus according to claim 21, wherein the distribution apparatus is further configured to save route configuration information, and the route configuration information is used to record a correspondence between each storage unit and the home storage controller configured for each storage unit; and
the processing unit is configured to search the route configuration information according to the identifier of the to-be-read/written storage unit and carried in the IO read/write instruction, to determine the home storage controller corresponding to the to-be-read/written storage unit.

### Claim 23 (depends on 22)

The distribution apparatus according to claim 22, wherein the processing unit is further configured to:
receive information about a storage unit homed by each storage controller and initially configured in a system; and
establish the route configuration information according to the received information about the storage unit homed by each storage controller and initially configured in the system.

### Claim 24 (depends on 23)

The distribution apparatus according to claim 23, wherein the processing unit is further configured to:
receive updated information about the storage unit homed by each storage controller; and
update the route configuration information according to the received updated information about the storage unit homed by each storage controller.

### Claim 25 (depends on 21)

The distribution apparatus according to claim 21, wherein the processing unit is further configured to:
receive a storage address sent by the home storage controller corresponding to the to-be-read/written storage unit; and
forward the received storage address to the host device, so that the host device operates, according to the storage address, data corresponding to the IO read/write instruction,
wherein the storage address is allocated from a cache or memory of the home storage controller for the IO read/write instruction after the home storage controller corresponding to the to-be-read/written storage unit receives the IO read/write instruction.

### Claim 26 (depends on 21)

The distribution apparatus according to claim 21, wherein a Peripheral Component Interconnect Express (PCIe) link is used for connection between the processing unit and the switch unit and between the host interaction unit and the switch unit, and the processing unit is configured as a PCIe root complex of the host interaction unit and each back-end interface;
the host interaction unit is configured to receive the IO read/write instruction sent by the host device, and send the IO read/write instruction to the PCIe root complex using the PCIe link; and
the processing unit, as the PCIe root complex, is configured to receive the IO read/write instruction sent by the host interaction unit using the PCIe link.

### Claim 27 (depends on 21)

The distribution apparatus according to claim 21, wherein each back-end interface is connected, using a Peripheral Component Interconnect Express (PCIe) none transparent bridge, to each storage controller connected in a one-to-one correspondence manner;
the processing unit is configured to:
after receiving the IO read/write instruction, determine the back-end interface connected to the home storage controller corresponding to the to-be-read/written storage unit;
fill a destination address of the IO read/write instruction as an address of the back-end interface connected to the home storage controller corresponding to the to-be-read/written storage unit; and
send the IO read/write instruction using the PCIe link; and
the back-end interface connected to the home storage controller corresponding to the to-be-read/written storage unit is configured to:
receive the IO read/write instruction;
map the destination address of the IO read/write instruction to an address of the home storage controller corresponding to the to-be-read/written storage unit; and
send the IO read/write instruction using the PCIe none transparent bridge.

### Claim 28

A data distribution method applied in a storage system comprising a distribution apparatus and a storage device, the storage device comprises at least two storage controllers and multiple storage units, each storage unit is configured with any one of the at least two storage controllers as a home storage controller, the distribution apparatus comprises a front-end interface and at least two back-end interfaces, the front-end interface is configured to connect to a host device, and each back-end interface is connected to each storage controller in a one-to-one correspondence manner; the method comprising:
receiving, by the distribution apparatus using the front-end interface, an IO read/write instruction sent by the host device, wherein the IO read/write instruction carries an identifier of a to-be-read/written storage unit;
determining, by the distribution apparatus, a home storage controller of the to-be-read/written storage unit according to the identifier of the to-be-read/written storage unit; and
forwarding, by the distribution apparatus, the IO read/write instruction to the home storage controller of the to-be-read/written storage unit using a back-end interface connected to the home storage controller of the to-be-read/written storage unit.

### Claim 29 (depends on 28)

The method according to claim 28, wherein the method further comprises:
receiving, by the distribution apparatus, a storage address sent by the home storage controller of the to-be-read/written storage unit, wherein the storage address is allocated from a cache or memory of the home storage controller for the IO read/write instruction after the home storage controller corresponding to the to-be-read/written storage unit receives the IO read/write instruction; and
forwarding, by the distribution apparatus, the storage address to the host device, so that the host device operates, according to the storage address, data corresponding to the IO read/write instruction.

### Claim 30 (depends on 29)

The method according to claim 29, wherein the IO read/write instruction is a write operation instruction, and after the forwarding, by the distribution apparatus, the storage address to the host device, the method further comprises:
receiving, by the distribution apparatus, a write operation end instruction sent by the host device, wherein the write operation end instruction is used to indicate that the host device has written, into the storage address in a direct memory access (DMA) manner, all data corresponding to the write operation instruction; and
forwarding, by the distribution apparatus, the write operation end instruction to the home storage controller of the to-be-read/written storage unit.

### Claim 31 (depends on 29)

The method according to claim 29, wherein the IO read/write instruction is a read operation instruction, and after the forwarding, by the distribution apparatus, the storage address to the host device, the method further comprises:
receiving, by the distribution apparatus, a read operation end instruction sent by the home storage controller of the to-be-read/written storage unit, wherein the read operation end instruction is used to indicate that the home storage controller of the to-be-read/written storage unit has sent, to the host device in a direct memory access (DMA) manner, all data corresponding to the read operation instruction; and
forwarding, by the distribution apparatus, the read operation end instruction to the host device.

### Claim 32 (depends on 28)

The method according to claim 28, wherein the determining, by the distribution apparatus, a home storage controller of the to-be-read/written storage unit according to the identifier of the to-be-read/written storage unit comprises:
searching, by the distribution apparatus, route configuration information according to the identifier of the to-be-read/written storage unit and carried in the IO read/write instruction, to determine the home storage controller corresponding to the to-be-read/written storage unit, wherein the route configuration information is used to record a correspondence between each storage unit and the home storage controller configured for each storage unit.

### Claim 33 (depends on 32)

The method according to claim 32, wherein before the receiving, by the distribution apparatus using the front-end interface, an IO read/write instruction sent by the host device, the method further comprises:
receiving, by the distribution apparatus, information about a storage unit homed by each storage controller and initially configured in the system, and establishing the route configuration information according to the received information about the storage unit homed by each storage controller and initially configured in the system.

### Claim 34 (depends on 33)

The method according to claim 33, wherein after the establishing the route configuration information, the method further comprises:
receiving, by the distribution apparatus, updated information about the storage unit homed by each storage controller, and updating the route configuration information according to the received updated information about the storage unit homed by each storage controller.