System for allocation of network resources for executing large language model (LLM) tasks

A system, the system comprising:

• a processing device;
• a non-transitory storage device containing instructions that, when executed by the processing device, cause the processing device to:
• receive a large language model (LLM) task and an input specifying information associated with execution of the LLM task, wherein the input comprises at least a data parallelism parameter, a pipeline parallelism parameter, and a communication pattern;
• determine a plurality of hosts based on at least the parallelism parameter and the communication pattern;
• segment the execution of the LLM task into a plurality of pipelines based on at least the data parallelism parameter;
• segment each pipeline into a plurality of pipeline stages based on the pipeline parallelism parameter;
• allocate the plurality of pipelines and the plurality of pipeline stages among the plurality of hosts, wherein the plurality of hosts is interconnected for data portion communication and pipeline communication;
• determine a plurality of switches based on the plurality of hosts;
• determine, based on the data parallelism parameter, a first set of optical circuit connections for each switch to facilitate the data portion communication between the plurality of hosts across the plurality of switches;
• determine, based on the pipeline parallelism parameter, a second set of optical circuit connections for each switch to facilitate the pipeline communication between the plurality of hosts across the plurality of switches;
• operatively couple the plurality of switches using the first set of optical circuit connections and the second set of optical circuit connections;
• dynamically configure a network point of delivery (POD) by operatively coupling the plurality of hosts to the plurality of switches using the first set of optical circuit connections and the second set of optical circuit connections;
• and
• trigger execution of the LLM task using the network POD,
• wherein a count of the first set of optical circuit connections and a count of the second set of optical circuit connections is determined to satisfy a full-bisection bandwidth requirement.

A computer program product, the computer program product comprising a non-transitory computer-readable medium comprising code configured to cause an apparatus to:

• receive a large language model (LLM) task and an input specifying information associated with execution of the LLM task, wherein the input comprises at least a data parallelism parameter, a pipeline parallelism parameter, and a communication pattern;
• determine a plurality of hosts based on at least the parallelism parameter and the communication pattern;
• segment the execution of the LLM task into a plurality of pipelines based on at least the data parallelism parameter;
• segment each pipeline into a plurality of pipeline stages based on the pipeline parallelism parameter;
• allocate the plurality of pipelines and the plurality of pipeline stages among the plurality of hosts, wherein the plurality of hosts is interconnected for data portion communication and pipeline communication;
• determine a plurality of switches based on the plurality of hosts;
• determine, based on the data parallelism parameter, a first set of optical circuit connections for each switch to facilitate the data portion communication between the plurality of hosts across the plurality of switches;
• determine, based on the pipeline parallelism parameter, a second set of optical circuit connections for each switch to facilitate the pipeline communication between the plurality of hosts across the plurality of switches;
• operatively couple the plurality of switches using the first set of optical circuit connections and the second set of optical circuit connections;
• dynamically configure a network point of delivery (POD) by operatively coupling the plurality of hosts to the plurality of switches using the first set of optical circuit connections and the second set of optical circuit connections;
• and
• trigger execution of the LLM task using the network POD,
• wherein a count of the first set of optical circuit connections and a count of the second set of optical circuit connections is determined to satisfy a full-bisection bandwidth requirement.