Cross-Domain Memory Infrastructure Between NVLink Networks and PCIe Fabrics via Protocol and Address Translations

Abstract

Training and inference of large-scale AI models, including large language models (LLMs), mixture-of-experts (MoE) models, multi-modal models, and diffusion models, require GPU clusters interconnected via NVLink to access memory, storage, and networking resources via PCIe, optionally including transferring model weights, key-value cache entries, and activation tensors. A computer, coupled between a first interface communicating according to an NVLink-based protocol with a first entity and a second interface communicating according to a PCIe protocol with a second entity, translates between NVLink-based PDUs and PCIe TLPs, including PCIe UIO TLPs. The translations may include translating physical addresses between address spaces associated with the respective protocols, translating between NVLink-based request types and PCIe TLP types, and translating identifier fields such as Tags, SourceIDs, DestinationIDs, and RequesterIDs. The computer may further translate between Flit Mode and Non-Flit Mode, and may be implemented as a semiconductor device, an active cable, or a chiplet.

Dependencies are inferred from the claim text.