# US 10,310,760 B1 · Layering communication fabric protocols > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,310,760 B1 - Document ID: US10310760B1 - Status: Granted - Kind: B1 - Application number: 16/175,316 - Filed: 2018-10-30 - Granted: 2019-06-04 - Earliest claimed: 2018-05-21 - Assignees: Pure Storage, Inc. - Applicants: Pure Storage, Inc. - Inventors: Roland Dreier; Steven Hodgson; Ronald Karr - Web page: https://patentgrep.com/patent/US10310760B1 ## CPC - G06F 3/0631: {by allocating resources to storage systems} - G06F 3/061: {Improving I/O performance} - G06F 3/067: {Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]} - G06F 13/42: Bus transfer protocol, e.g. handshake; Synchronisation - G06F 16/1824: {implemented using Network-attached Storage [NAS] architecture (distributed or networked storage systems G06F3/067; protocols for distributed storage of data in a network H04L67/1097)} - G06F 16/1827: {Management specifically adapted to NAS (management of storage area networks [SAN] G06F3/067)} - G06F 30/34: for reconfigurable circuits, e.g. field programmable gate arrays [FPGA] or programmable logic devices [PLD] - H03K 19/017581: {programmable} - H04L 67/06: specially adapted for file transfer, e.g. file transfer protocol [FTP] - H04L 67/1095: Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes - H04L 67/1097: for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS] - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H03K: PULSE TECHNIQUE (measuring pulse characteristics G01R; modulating sinusoidal oscillations with pulses H03C; transmission of digital information H04L; discriminator circuits detecting phase difference between two signals by counting or integrating cycles of oscillation H03D3/04; automatic control, starting, synchronisation or stabilisation of generators of electronic oscillations or pulses where the type of generator is irrelevant or unspecified H03L; coding, decoding or code conversion, in general H03M) - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract Layering communication fabric protocols, including: allocating storage resources on an initiating storage system for a data transfer, wherein a command protocol to implement the data transfer specifies an exchange of multiple messages; receiving, from a target storage system and into the allocated storage resources at the initiating storage system, data within a single message corresponding to the command protocol that specifies an exchange of multiple messages to implement the data transfer; and in response to receiving the data within the single message corresponding to the command protocol that specifies an exchange of multiple messages to implement the data transfer, generating a status message indicating receipt of the data. ## Claims ### Claim 1 A method of layering communication fabric protocols, the method comprising: allocating storage resources on an initiating storage system for a data transfer, wherein a command protocol to implement the data transfer specifies an exchange of multiple messages; receiving, from a target storage system and into the allocated storage resources at the initiating storage system, data within a single message corresponding to the command protocol that specifies an exchange of multiple messages to implement the data transfer; and in response to receiving the data within the single message corresponding to the command protocol that specifies an exchange of multiple messages to implement the data transfer, generating a status message indicating receipt of the data. ### Claim 2 (depends on 1) The method of claim 1, wherein the data transfer is performed after receiving the single message without receiving all messages comprising the command protocol. ### Claim 3 (depends on 1) The method of claim 1, wherein the storage resources are allocated for a read message from the initiating storage system to the target storage system, and wherein the initiating storage system does not expect data returned from the target storage system in response to the read message. ### Claim 4 (depends on 1) The method of claim 1, wherein the storage resources allocated comprise multiple memory buffers. ### Claim 5 (depends on 4) The method of claim 4, wherein a size for the memory buffer is allocated in dependence upon one or more of: a communication link bandwidth, network latency, or a payload specification for the command protocol. ### Claim 6 (depends on 4) The method of claim 4, wherein a size of one memory buffer or a combined size of more than one memory buffer of the multiple buffers is based upon a maximum quantity of data that can be in transit between the initiating storage system and the target storage system at a given time. ### Claim 7 (depends on 1) The method of claim 1, wherein the command protocol is non-volatile memory express (NVMe), wherein a communication fabric linking the source storage system to the target storage system implements Fibre Channel, wherein the data transfer is performed using an open exchange created by the target storage system sending an NVMe SQE (submission queue entry) message, and wherein the status message is an NVMe CQE (completion queue entry) message. ### Claim 8 An apparatus for layering communication fabric protocols, the apparatus comprising a computer processor, a computer memory operatively coupled to the computer processor, the computer memory having disposed within it computer program instructions that, when executed by the computer processor, cause the apparatus to carry out the steps of: allocating storage resources on an initiating storage system for a data transfer, wherein a command protocol to implement the data transfer specifies an exchange of multiple messages; receiving, from a target storage system and into the allocated storage resources at the initiating storage system, data within a single message corresponding to the command protocol that specifies an exchange of multiple messages to implement the data transfer; and in response to receiving the data within the single message corresponding to the command protocol that specifies an exchange of multiple messages to implement the data transfer, generating a status message indicating receipt of the data. ### Claim 9 (depends on 8) The apparatus of claim 8, wherein the data transfer is performed after receiving the single message without receiving all messages comprising the command protocol. ### Claim 10 (depends on 8) The apparatus of claim 8, wherein the storage resources are allocated for a read message from the initiating storage system to the target storage system, and wherein the initiating storage system does not expect data returned from the target storage system in response to the read message. ### Claim 11 (depends on 8) The apparatus of claim 8, wherein the storage resources allocated comprise multiple memory buffers. ### Claim 12 (depends on 11) The apparatus of claim 11, wherein a size for the memory buffer is allocated in dependence upon one or more of: a communication link bandwidth, network latency, or a payload specification for the command protocol. ### Claim 13 (depends on 11) The apparatus of claim 11, wherein a size of one memory buffer or a combined size of more than one memory buffer of the multiple buffers is based upon a maximum quantity of data that can be in transit between the initiating storage system and the target storage system at a given time. ### Claim 14 (depends on 8) The apparatus of claim 8, wherein the command protocol is non-volatile memory express (NVMe), wherein a communication fabric linking the source storage system to the target storage system implements Fibre Channel, wherein the data transfer is performed using an open exchange created by the target storage system sending an NVMe SQE (submission queue entry) message, and wherein the status message is an NVMe CQE (completion queue entry) message. ### Claim 15 A computer program product for layering communication fabric protocols, the computer program product disposed upon a computer readable medium, the computer program product comprising computer program instructions that, when executed, cause a computer to carry out the steps of: allocating storage resources on an initiating storage system for a data transfer, wherein a command protocol to implement the data transfer specifies an exchange of multiple messages; receiving, from a target storage system and into the allocated storage resources at the initiating storage system, data within a single message corresponding to the command protocol that specifies an exchange of multiple messages to implement the data transfer; and in response to receiving the data within the single message corresponding to the command protocol that specifies an exchange of multiple messages to implement the data transfer, generating a status message indicating receipt of the data. ### Claim 16 (depends on 15) The computer program product of claim 15, wherein the data transfer is performed after receiving the single message without receiving all messages comprising the command protocol. ### Claim 17 (depends on 15) The computer program product of claim 15, wherein the storage resources are allocated for a read message from the initiating storage system to the target storage system, and wherein the initiating storage system does not expect data returned from the target storage system in response to the read message. ### Claim 18 (depends on 15) The computer program product of claim 15, wherein the storage resources allocated comprise multiple memory buffers. ### Claim 19 (depends on 18) The computer program product of claim 18, wherein a size for the memory buffer is allocated in dependence upon one or more of: a communication link bandwidth, network latency, or a payload specification for the command protocol. ### Claim 20 (depends on 18) The computer program product of claim 18, wherein a size of one memory buffer or a combined size of more than one memory buffer of the multiple buffers is based upon a maximum quantity of data that can be in transit between the initiating storage system and the target storage system at a given time.