Translating Between UALink UPLI Requests and CXL.cache Messages for Scalable AI Inference Memory Expansion Beyond the Accelerator Memory Wall

An apparatus comprising:

• a first interface configured to communicate according to an Ultra Accelerator Link Protocol Level Interface (UPLI);
• a second interface configured to communicate according to CXL.cache, wherein CXL denotes Compute Express Link;
• and
• a computer coupled to the first interface and the second interface, the computer configured to:
• translate a UPLI request comprising a read command, received via the first interface, to a CXL.cache Device-to-Host (D2H) request comprising a read opcode, and send the CXL.cache D2H request via the second interface;
• and
• translate a CXL.cache Host-to-Device (H2D) data message, received via the second interface, to a UPLI read response, and send the UPLI read response via the first interface.

A method comprising:

• executing, by an accelerator comprising local memory, an inference workload on a neural network model;
• sending, by the accelerator, an Ultra Accelerator Link Protocol Level Interface (UPLI) request comprising a read command for model data stored in a host memory of a host;
• translating, by a Resource Provisioning Unit (RPU), the UPLI request to a CXL.cache Device-to-Host (D2H) request comprising a read opcode, wherein CXL denotes Compute Express Link;
• sending the CXL.cache D2H request to the host;
• receiving, from the host, a CXL.cache Host-to-Device (H2D) data message comprising the model data;
• translating, by the RPU, the CXL.cache H2D data message to a UPLI read response comprising the model data;
• and
• sending the UPLI read response to the accelerator.

A system comprising:

• an accelerator comprising local memory and configured to communicate according to an Ultra Accelerator Link Protocol Level Interface (UPLI);
• a host comprising host memory and configured to communicate according to CXL.cache, wherein CXL denotes Compute Express Link;
• and
• a Resource Provisioning Unit (RPU) coupled between the accelerator and the host, the RPU comprising a first interface configured to communicate with the accelerator according to UPLI and a second interface configured to communicate with the host according to CXL.cache;
• wherein the RPU is configured to:
• translate a UPLI request comprising a read command, received from the accelerator via the first interface, to a CXL.cache Device-to-Host (D2H) request comprising a read opcode, and send the CXL.cache D2H request to the host via the second interface;
• and
• translate a CXL.cache Host-to-Device (H2D) data message, received from the host via the second interface, to a UPLI read response, and send the UPLI read response to the accelerator via the first interface;
• whereby the accelerator accesses the host memory via the RPU.