# US 11,615,038 B2 · Data through gateway > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,615,038 B2 - Document ID: US11615038B2 - Status: Granted - Kind: B2 - Application number: 16/235,612 - Filed: 2018-12-28 - Published: 2020-01-09 - Granted: 2023-03-28 - Earliest claimed: 2018-07-04 - Assignees: Graphcore Limited - Applicants: Graphcore Limited - Inventors: Ola Tørudbakken; Brian Manula; Harald Høeg - Web page: https://patentgrep.com/patent/US11615038B2 ## CPC - G06F 13/1673: {using buffers} - G06F 9/3851: {from multiple instruction streams, e.g. multistreaming} - G06F 9/5027: {the resource being a machine, e.g. CPUs, Servers, Terminals} - G06F 9/5044: {considering hardware capabilities} - G06F 9/522: {Barrier synchronisation} - G06F 9/54: Interprogram communication - G06F 12/0813: with a network or matrix configuration - G06F 12/0868: Data transfer between cache memory and other subsystems, e.g. storage devices or host systems - G06F 13/1689: {Synchronisation and timing concerns (synchronisation on a memory bus G06F13/4234)} - G06F 2209/509: Offload - H04L 12/5601: {Transfer mode dependent, e.g. ATM} - H04L 2012/5618: {Bridges, gateways [GW] or interworking units [IWU]} - H04L 2012/5625: {Operations, administration and maintenance [OAM]} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract A gateway for use in a computing system to interface a host with the subsystem for acting as a work accelerator to the host, the gateway having: an accelerator interface for connection to the subsystem to enable transfer of batches of data between the subsystem and the gateway; a data connection interface for connection to external storage for exchanging data between the gateway and storage; a gateway interface for connection to at least one second gateway; a memory interface connected to a local memory associated with the gateway; and a streaming engine for controlling the streaming of batches of data into and out of the gateway in response to pre-compiled data exchange synchronisation points attained by the subsystem, wherein the streaming of batches of data are selectively via at least one of the accelerator interface, data connection interface, gateway interface and memory interface. ## Claims ### Claim 1 A gateway for use in a computing system to interface a host with a subsystem for acting as a first work accelerator to the host, the gateway having: an accelerator interface for connection to the subsystem to enable transfer of batches of data between the subsystem and the gateway when a predetermined data exchange synchronisation point in compiled code executed by the subsystem is reached; a data connection interface for connection to external storage for exchanging data between the gateway and storage; a gateway interface for connection to at least one second gateway for interfacing the host with a further subsystem for acting as a second work accelerator to the host, wherein the gateway comprises at least one processor configured to execute instructions to, in advance of the data exchange synchronisation point being reached in the compiled code executed by the subsystem, retrieve one or more batches of data from the at least one second gateway, wherein the gateway is associated with a local memory configured to store the one or more batches of data received from the at least one second gateway such that the one or more batches of data are available for provision to the subsystem via the accelerator interface when the data exchange synchronisation point in the compiled code executed by the subsystem is reached. ### Claim 2 (depends on 1) A gateway as claimed in claim 1, wherein one or more of the batches of data are transferred from the local memory to the subsystem in response to the data exchange synchronisation point in the compiled code executed by the subsystem being reached. ### Claim 3 (depends on 1) A gateway as claimed in claim 1, wherein the at least one processor is configured to, render available the one or more batches of data at one or more indicated storage locations in the local memory to be pulled by the subsystem from the local memory when the data exchange synchronisation point in the compiled code executed by the subsystem is reached. ### Claim 4 (depends on 1) A gateway as claimed in claim 1, wherein the least one processor is configured to execute instructions to push the one or more batches of data from the local memory to the subsystem in response to receipt of an indication that the data exchange synchronisation point has been reached by the subsystem. ### Claim 5 (depends on 1) A gateway as claimed in claim 1, wherein the at least one processor is configured to stream the one or more batches of data from the at least one second gateway into the local memory prior to the predetermined data exchange synchronisation point being reached in the compiled code executed by the subsystem. ### Claim 6 (depends on 1) A gateway as claimed in claim 1, wherein the at least one processor is configured to stream a further one or more batches of data from the external storage into the local memory prior to the predetermined data exchange synchronisation point being reached in the compiled code executed by the subsystem, wherein the further one or more of the batches of data are transferred from the local memory to the subsystem in response to the data exchange synchronisation point in the compiled code executed by the subsystem being reached. ### Claim 7 (depends on 4) A gateway as claimed in claim 4, wherein the at least one processor is configured to operate as a streaming engine comprising a data mover engine and a memory management engine, the data mover engine and memory management engine configured to execute instructions in coordination from work descriptors, the memory management engine configured to execute instructions from a work descriptor to stream the one or more batches of data into the local memory prior to the one of the predetermined data exchange synchronisation point being reached in the compiled code executed by the subsystem, the data mover engine configured to execute from the work descriptor, the instructions to push the one or more batches of data from the local memory to the subsystem in response to receipt of the indication that the data exchange synchronisation point has been reached by the subsystem. ### Claim 8 (depends on 1) A gateway as claimed in claim 1, wherein the gateway is configured to: upon the subsystem reaching the data exchange synchronisation point in the execution of the compiled code, receive, at the accelerator interface, a further one or more of the batches of data; and store said further one or more batches of data in the local memory. ### Claim 9 (depends on 8) A gateway as claimed in claim 8, wherein the at least one processor is configured to operate as a streaming engine comprising a data mover engine and a memory management engine, the data mover engine and memory management engine configured to execute instructions in coordination from work descriptors, the memory management engine configured to execute instructions from a work descriptor to stream the one or more batches of data into the local memory prior to the one of the predetermined data exchange synchronisation point being reached in the compiled code executed by the subsystem, the data mover engine configured to execute from the work descriptor, the instructions to push the one or more batches of data from the local memory to the subsystem in response to receipt of an indication that the data exchange synchronisation point has been reached by the subsystem, wherein the memory management engine is configured to execute instructions from the work descriptor to stream the further one or more of the batches of data to the external storage. ### Claim 10 (depends on 1) A gateway as claimed in claim 1, wherein the gateway is configured to receive from the subsystem, a data packet when the data exchange synchronisation point in the compiled code executed by the subsystem is reached. ### Claim 11 (depends on 10) A gateway as claimed in claim 10, configured to: examine an address of the data packet to determine whether the data packet is addressed to a resource associated with the gateway; and in response to determining that the data packet is addressed to resources associated with the gateway forward the data packet to said resource, wherein the resource associated with the gateway comprises one or more of: a second subsystem for acting as the second work accelerator to the host; and a memory of the gateway. ### Claim 12 (depends on 10) A gateway as claimed in claim 10, configured to: look up in a routing table, a port of the gateway interface corresponding to an address indicated in the data packet; and forward the data packet over the port of the gateway. ### Claim 13 (depends on 12) A gateway as claimed in claim 12, wherein the address indicates an address of an accelerator associated with the second gateway, wherein the gateway is configured to forward the data packet over the gateway interface to the second gateway. ### Claim 14 (depends on 11) A gateway as claimed in claim 11, wherein the address is an address in a global address space encompassing a plurality of accelerators and gateways in a scaled system. ### Claim 15 (depends on 1) A gateway as claimed in claim 1, wherein the external storage comprises at least one of: a storage of the host; local storage and a network attached storage. ### Claim 16 (depends on 12) A gateway as claimed in claim 12, wherein the gateway interface comprises multiple ports, each port connected to a corresponding gateway of multiple gateways. ### Claim 17 (depends on 14) A gateway as claimed in claim 14 wherein the accelerators and gateways form part of a common synchronisation zone and are each configured to generate a synchronisation acknowledgement responsive to receiving an indication of the one of the data exchange synchronisation point when a respective condition is met. ### Claim 18 A method of interfacing a gateway with a first work accelerator for a host system, the method comprising: when a data exchange synchronisation point in compiled code executed by the first work accelerator is reached, communicating batches of data between the gateway and the first work accelerator, by an accelerator interface; communicating between the gateway and external storage, by a data connection interface; communicating, by a gateway interface, between the gateway and a second gateway, for interfacing the host system with a second work accelerator; in advance of a data exchange synchronisation point being reached in compiled code executed by the first work accelerator, receiving a first batch of data from the second gateway; storing in local memory, the first batch of data such that the first batch of data is available for provision to the first work accelerator via the accelerator interface when the data exchange synchronisation point in the compiled code executed by the first work accelerator is reached. ### Claim 19 (depends on 18) The method of claim 18, wherein the method comprises: transferring the data from the local memory to the first work accelerator in response to the data exchange synchronization point in the compiled code executed by the first work accelerator being reached. ### Claim 20 (depends on 18) The method of claim 18, wherein the method comprises: streaming the data from the second gateway into the local memory prior to the data exchange synchronization point in the compiled code executed by the first work accelerator being reached. ### Claim 21 (depends on 18) The method of claim 18, wherein the method comprises: executing instructions from a work descriptor to stream the data into the local memory prior to the data exchange synchronization point in the compiled code executed by the first work accelerator being reached. ### Claim 22 (depends on 18) The method of claim 18, wherein the method comprises: executing, from a work descriptor, instructions to push the data from the local memory to the first work accelerator in response to the data exchange synchronization point in the compiled code executed by the first work accelerator being reached. ### Claim 23 (depends on 18) The method of claim 18, wherein the method comprises: receiving, at the accelerator interface, further data; storing the further data into the local memory; and streaming the further data from the local memory to external storage. ### Claim 24 (depends on 18) The method of claim 18, wherein the method comprises: determining that a data packet of the data is addressed to a resource associated with the gateway by examining an address of the data packet; and forwarding the data packet to the resource, wherein the resource includes an item selected from the list consisting of the second work accelerator and a memory of the gateway. ### Claim 25 (depends on 18) The method of claim 18, wherein the method comprises: looking up in a routing table, a port of the gateway interface corresponding to an address indicated in a data packet of the data; and forwarding the data packet over the port of the gateway. ### Claim 26 A system comprising: a first work accelerator; and a first gateway for interfacing a host with the first work accelerator; and a second gateway for interfacing the host with a second work accelerator, the first gateway including: an accelerator interface providing a connection to the first work accelerator for transferring data when a data exchange synchronisation point in compiled code executed by the first work accelerator is reached; a data connection interface providing a connection to external storage; a gateway interface providing a connection to the second gateway, wherein the first gateway is configured to, in advance of the data exchange synchronisation point being reached in the compiled code executed by the first work accelerator, receive a batch of data from the second gateway, wherein the first gateway is associated with a local memory configured to store the batch of data received from the second gateway such that the batch of data is available for provision to the first work accelerator via the accelerator interface when the data exchange synchronisation point in the compiled code executed by the first work accelerator is reached.