Data transform acceleration using metadata stored in accelerator memory

A host comprising:

• data processing hardware;
• and
• memory hardware in communication with the data processing hardware, the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations comprising:
• generating a container data structure in a data transform accelerator, the data transform accelerator in data communication with the host;
• generating input data in the memory hardware;
• generating metadata in the data transform accelerator or spread out in the memory of data transform accelerator and host memory hardware;
• reserving output buffer in the memory hardware;
• generating a first data transform command in the memory hardware, the first data transform command associated with the input data and the metadata;
• and
• updating the container data structure with an address of the first data transform command,
• wherein the address of the first data transform command is accessible by a data transform accelerator,
• wherein accessing the address of the first data transform command by the data transform accelerator causes the data transform accelerator to obtain the input data, to perform one or more data transform operations on the input data based on the metadata, and to transmit output data to the output buffer, and
• wherein the output data is the input data after being transformed by the one or more data transform operations.