Apparatus and method for large language model (llm) reinforcement learning with heterogeneous processors

A machine-readable medium having program code stored thereon which, when executed by one or more processors, is to cause the one or more processors to perform operations, comprising:

• registering each training engine instance of a training engine to process a specific shard of model weights of a large language model (LLM), each training engine instance executed by a first one or more processors;
• generating metadata in response to registering, the metadata including remote direct memory access (RDMA) addresses for corresponding shard memory locations;
• performing, by each rollout engine worker of a rollout engine, RDMA read operations using the metadata to fetch corresponding weight shards, the rollout engine executed by a second one or more processors heterogeneous with respect to the first one or more processors;
• updating weights of a local model in accordance with the weight shards;
• and
• performing LLM inference operations based on the local model to generate corresponding training data samples.