Gateway processing

A method for interfacing a subsystem with a host system via a gateway, the subsystem acting as a work accelerator to the host system, the method comprising:

• executing compiled code on the subsystem to i) during a compute phase of the subsystem, process data provided by the host system and ii) during an exchange phase of the subsystem, exchange data with the gateway, wherein the exchange phase is separated from the compute phase by a barrier synchronisation indicated in the compiled code executed by the subsystem;
• and
• receiving at the gateway, data provided by the host system for transfer to the subsystem during the exchange phase;
• following receipt of the data at the gateway, and in advance of the barrier synchronisation, performing data preparation processing on the data to produce a first set of prepared data;
• and
• making said first set of prepared data available to be transferred to the subsystem during the exchange phase, which follows the barrier synchronisation,
• wherein the gateway comprises a gateway interface for connecting to a second gateway, and wherein the method further comprises:
• processing the one or more batches of data to produce a plurality of sets of data;
• applying data preparation operations to at least some of the plurality of sets of data with different settings applied for different ones of the plurality of sets of data to produce a plurality of sets of prepared data, the plurality of sets of prepared data including the first set of prepared data and a second set of prepared data;
• and
• transferring the second set of prepared data to the second gateway.

A non-transitory computer readable medium storing:

• a first set of computer readable instructions for execution by at least one processor of a subsystem for acting as a work accelerator to a host system;
• and
• a second set of computer readable instructions for execution by at least one processor of a gateway for interfacing the subsystem with the host system, wherein the first set of computer readable instructions and the second set of computer readable instructions are configured to, when executed, cause a method to be performed, the method comprising:
• at the subsystem:
• i) during a compute phase of the subsystem, processing data provided by the host system and ii) during an exchange phase of the subsystem, exchanging data with the gateway, wherein the exchange phase is separated from the compute phase by a barrier synchronisation indicated in the first set of computer readable instructions executed by the subsystem;
• and
• receiving at the gateway, data provided by the host system for transfer to the subsystem during the exchange phase;
• following receipt of the data at the gateway, and in advance of the barrier synchronisation, performing data preparation processing on the data to produce a first set of prepared data;
• and
• making said first set of prepared data available to be transferred to the subsystem during the exchange phase, which follows the barrier synchronisation;
• wherein the gateway comprises a gateway interface for connection to a second gateway, and wherein the method further comprises:
• processing the one or more batches of data to produce a plurality of sets of data;
• applying data preparation operations to at least some of the plurality of sets of data with different settings applied for different ones of the plurality of sets of data to produce a plurality of sets of prepared data, the plurality of sets of prepared data including the first set of prepared data and a second set of prepared data;
• and
• transferring the second set of prepared data to the second gateway.