# US 2025/0211548 A1 · System for allocation of network resources for executing large language model (llm) tasks

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0211548 A1
- Document ID: US20250211548A1
- Status: Published application
- Kind: A1
- Application number: 18/416,028
- Filed: 2024-01-18
- Published: 2025-06-26
- Earliest claimed: 2023-12-20
- Assignees: Mellanox Technologes, Ltd.
- Applicants: Mellanox Technologes, Ltd.
- Inventors: Ioannis (Giannis) PATRONAS; Nikolaos TERZENIDIS; Eitan ZAHAVI; Paraskevas BAKOPOULOS; Zsolt-Alon WERTHEIMER; Dimitrios SYRIVELIS; Prethvi Ramesh KASHINKUNTI; Louis Bennie CAPPS, JR.; Julie Irene Marcelle BERNAUER; Elad MENTOVICH
- Web page: https://patentgrep.com/patent/US20250211548A1

## CPC

- H04L 47/808: {User-type aware}
- H04L 47/781: {Centralised allocation of resources}
- H04L 47/827: {Aggregation of resource allocation or reservation requests}
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

Systems, computer program products, and methods are described herein for allocation of network resources for executing large language model (LLM) tasks. An example system receives an LLM task and an input specifying information associated with execution of the LLM task, wherein the input comprises at least a parallelism parameter and a communication pattern; determines a plurality of hosts based on at least the parallelism parameter and the communication pattern; determines a plurality of switches based on the plurality of hosts; operatively couples the plurality of hosts to the plurality of switches to configure a network point of delivery (POD); and triggers execution of the LLM task using the network POD.

## Claims

### Claim 1

A method for allocation of network resources for executing large language model (LLM) tasks, the method comprising:
receiving an LLM task and an input specifying information associated with execution of the LLM task, wherein the input comprises at least a parallelism parameter and a communication pattern;
determining a plurality of hosts based on at least the parallelism parameter and the communication pattern;
determining a plurality of switches based on the plurality of hosts;
operatively coupling the plurality of hosts to the plurality of switches to dynamically configure a network point of delivery (POD); and
triggering execution of the LLM task using the network POD.

### Claim 2 (depends on 1)

The method of claim 1, wherein the parallelism parameter comprises at least one of a data parallelism parameter or a pipeline parallelism parameter, wherein the data parallelism parameter indicates a number of pipelines for executing the LLM task, wherein each pipeline represents a data partition, wherein the pipeline parallelism parameter indicates a number of pipeline stages for each pipeline, wherein each pipeline stage represents a portion of the corresponding data partition.

### Claim 3 (depends on 2)

The method of claim 2, further comprising:
segmenting the execution of the LLM task into a plurality of pipelines based on the data parallelism parameter;
segmenting each pipeline into a plurality of pipeline stages based on the pipeline parallelism parameter; and
allocating the plurality of pipelines and the plurality of pipeline stages among the plurality of hosts.

### Claim 4 (depends on 3)

The method of claim 3, wherein the plurality of hosts is interconnected for data portion communication and pipeline communication, wherein the data portion communication is based on the communication pattern associated with the data parallelism parameter and the pipeline communication is based on the communication pattern associated with the pipeline parallelism parameter.

### Claim 5 (depends on 4)

The method of claim 4, further comprising:
determining a first set of optical circuit connections for each switch to facilitate the data portion communication between the plurality of hosts across the plurality of switches;
determining a second set of optical circuit connections for each switch to facilitate the pipeline communication between the plurality of hosts across the plurality of switches; and
operatively coupling the plurality of switches using the first set of optical circuit connections and the second set of optical circuit connections.

### Claim 6 (depends on 5)

The method of claim 5, wherein a count of the first set of optical circuit connections and a count of the second set of optical circuit connections is determined to satisfy a specific bandwidth requirement.

### Claim 7 (depends on 6)

The method of claim 6, wherein, in instances in which the specific bandwidth requirement is a full-bisection bandwidth requirement,
the count of the first set of optical circuit connections is greater than or equal to 2*ps*k, wherein ps is number of pipeline stages allocated to a subset of the plurality of hosts that are operatively coupled to each switch, wherein k is a fractional bandwidth requirement for each data portion communication in each direction relative to a total bandwidth of an optical circuit connection in the first set of optical circuit connections, and
the count of the second set of optical circuit connections is greater than or equal to 2*p*m, wherein p is the number of pipelines allocated to the subset of the plurality of hosts that are operatively coupled to each switch, and wherein m is a fractional bandwidth requirement for each pipeline communication in each direction relative to a total bandwidth of an optical circuit connection in the second set of optical circuit connections.

### Claim 8 (depends on 3)

The method of claim 3, wherein the plurality of hosts is operatively coupled to the same switch.

### Claim 9 (depends on 2)

The method of claim 2, wherein the communication pattern associated with the pipeline parallelism parameter comprises at least a point-to-point communication, and wherein the communication pattern associated with the data parallelism parameter comprises at least a reduction operation.

### Claim 10 (depends on 1)

The method of claim 1, wherein the network POD is configured based on a closed loop topology to allow the plurality of hosts to communicate with one another via the plurality of switches, wherein the closed loop topology comprises at least one of a ring topology or a torus topology.

### Claim 11 (depends on 1)

The method of claim 1, wherein the network POD is configured based on an in-network collective, wherein the in-network collective comprises at least a scalable hierarchical aggregation and reduction protocol (SHARP) model in which the network POD is configured by allocating a plurality of circuits from each switch such that an aggregate count of the plurality of switches is equal to an aggregate count of distinct reductions associated with the switch, thereby ensuring full bandwidth utilization, and constructing a network topology that includes designated root switches for facilitating the reductions.

### Claim 12 (depends on 1)

The method of claim 1, further comprising:
configuring a network structure with a plurality of network PODs;
determining a plurality of spine switches based on at least the plurality of network PODs;
interconnecting the plurality of network PODs via the plurality of spine switches; and
triggering the execution of the LLM task using the network structure.

### Claim 13

A system for allocation of network resources for executing large language model (LLM) tasks, the system comprising:
a processing device;
a non-transitory storage device containing instructions that, when executed by the processing device, cause the processing device to:
receive an LLM task and an input specifying information associated with execution of the LLM task, wherein the input comprises at least a parallelism parameter and a communication pattern;
determine a plurality of hosts based on at least the parallelism parameter and the communication pattern;
determine a plurality of switches based on the plurality of hosts;
operatively couple the plurality of hosts to the plurality of switches to dynamically configure a network POD; and
trigger execution of the LLM task using the network POD.

### Claim 14 (depends on 13)

The system of claim 13, wherein the parallelism parameter comprises at least one of a data parallelism parameter or a pipeline parallelism parameter, wherein the data parallelism parameter indicates a number of pipelines for executing the LLM task, wherein each pipeline represents a data partition, wherein the pipeline parallelism parameter indicates a number of pipeline stages for each pipeline, wherein each pipeline stage represents a portion of the corresponding data partition.

### Claim 15 (depends on 14)

The system of claim 14, wherein the instructions, when executed, cause the processing device to:
segment the execution of the LLM task into a plurality of pipelines based on the data parallelism parameter;
segment each pipeline into a plurality of pipeline stages based on the pipeline parallelism parameter; and
allocate the plurality of pipelines and the plurality of pipeline stages among the plurality of hosts.

### Claim 16 (depends on 15)

The system of claim 15, wherein the plurality of hosts is interconnected for data portion communication and pipeline communication, wherein the data portion communication is based on the communication pattern associated with the data parallelism parameter and pipeline communication is based on the communication pattern associated with the pipeline parallelism parameter.

### Claim 17 (depends on 16)

The system of claim 16, wherein the instructions, when executed by the processing device, further cause the processing device to:
determine a first set of optical circuit connections for each switch to facilitate the data portion communication between the plurality of hosts across the plurality of switches;
determine a second set of optical circuit connections for each switch to facilitate the pipeline communication between the plurality of hosts across the plurality of switches; and
operatively couple the plurality of switches using the first set of optical circuit connections and the second set of optical circuit connections.

### Claim 18 (depends on 17)

The system of claim 17, wherein a count of the first set of optical circuit connections and a count of the second set of optical circuit connections is determined to satisfy a specific bandwidth requirement.

### Claim 19 (depends on 18)

The system of claim 18, wherein, in instances in which the specific bandwidth requirement is a full-bisection bandwidth requirement,
the count of the first set of optical circuit connections is greater than or equal to 2*ps*k, wherein ps is number of pipeline stages allocated to a subset of the plurality of hosts that are operatively coupled to each switch, wherein k is a fractional bandwidth requirement for each data portion communication in each direction relative to a total bandwidth of an optical circuit connection in the first set of optical circuit connections, and
the count of the second set of optical circuit connections is greater than or equal to 2*p*m, wherein p is the number of pipelines allocated to the subset of the plurality of hosts that are operatively coupled to each switch, and wherein m is a fractional bandwidth requirement for each pipeline communication in each direction relative to a total bandwidth of an optical circuit connection in the second set of optical circuit connections.

### Claim 20 (depends on 15)

The system of claim 15, wherein the plurality of hosts is operatively coupled to the same switch.

### Claim 21 (depends on 13)

The system of claim 13, wherein the network POD is configured based on a closed loop topology to allow the plurality of hosts to communicate with one another via the plurality of switches, wherein the closed loop topology comprises at least one of a ring topology or a torus topology.

### Claim 22 (depends on 13)

The system of claim 13, wherein the network POD is configured based on an in-network collective, wherein the in-network collective comprises at least a scalable hierarchical aggregation and reduction protocol (SHARP) model in which the network POD is configured by allocating a plurality of circuits from each switch such that an aggregate count of the plurality of switches is equal to an aggregate count of distinct reductions associated with the switch, thereby ensuring full bandwidth utilization, and constructing a network topology that includes designated root switches for facilitating the reductions.

### Claim 23 (depends on 13)

The system of claim 13, wherein the instructions, when executed, cause the processing device to:
configure a network structure with a plurality of network PODs;
determine a plurality of spine switches based on at least the plurality of network PODs;
interconnect the plurality of network PODs via the plurality of spine switches; and
trigger the execution of the LLM task using the network structure.

### Claim 24

A computer program product for allocation of network resources for executing large language model (LLM) tasks, the computer program product comprising a non-transitory computer-readable medium comprising code configured to cause an apparatus to:
receive an LLM task and an input specifying information associated with execution of the LLM task, wherein the input comprises at least a parallelism parameter and a communication pattern;
determine a plurality of hosts based on at least the parallelism parameter and the communication pattern;
determine a plurality of switches based on the plurality of hosts;
operatively couple the plurality of hosts to the plurality of switches to dynamically configure a network POD; and
trigger execution of the LLM task using the network POD.

### Claim 25 (depends on 24)

The computer program product of claim 24, wherein the parallelism parameter comprises at least one of a data parallelism parameter or a pipeline parallelism parameter, wherein the data parallelism parameter indicates a number of pipelines for executing the LLM task, wherein each pipeline represents a data partition, wherein the pipeline parallelism parameter indicates a number of pipeline stages for each pipeline, wherein each pipeline stage represents a portion of the corresponding data partition.

### Claim 26 (depends on 25)

The computer program product of claim 25, wherein the code further causes the apparatus to:
segment the execution of the LLM task into a plurality of pipelines based on the data parallelism parameter;
segment each pipeline into a plurality of pipeline stages based on the pipeline parallelism parameter; and
allocate the plurality of pipelines and the plurality of pipeline stages among the plurality of hosts.

### Claim 27 (depends on 26)

The computer program product of claim 26, wherein the plurality of hosts is interconnected for data portion communication and pipeline communication, wherein the data portion communication is based on the communication pattern associated with the data parallelism parameter and pipeline communication is based on the communication pattern associated with the pipeline parallelism parameter.

### Claim 28 (depends on 27)

The computer program product of claim 27, wherein the code further causes the apparatus to:
determine a first set of optical circuit connections for each switch to facilitate the data portion communication between the plurality of hosts across the plurality of switches;
determine a second set of optical circuit connections for each switch to facilitate the pipeline communication between the plurality of hosts across the plurality of switches; and
operatively couple the plurality of switches using the first set of optical circuit connections and the second set of optical circuit connections.

### Claim 29 (depends on 28)

The computer program product of claim 28, wherein a count of the first set of optical circuit connections and a count of the second set of optical circuit connections is determined to satisfy a specific bandwidth requirement.

### Claim 30 (depends on 29)

The computer program product of claim 29, wherein, in instances in which the specific bandwidth requirement is a full-bisection bandwidth requirement,
the count of the first set of optical circuit connections is greater than or equal to 2*ps*k, wherein ps is number of pipeline stages allocated to a subset of the plurality of hosts that are operatively coupled to each switch, wherein k is a fractional bandwidth requirement for each data portion communication in each direction relative to a total bandwidth of an optical circuit connection in the first set of optical circuit connections, and
the count of the second set of optical circuit connections is greater than or equal to 2*p*m, wherein p is the number of pipelines allocated to the subset of the plurality of hosts that are operatively coupled to each switch, and wherein m is a fractional bandwidth requirement for each pipeline communication in each direction relative to a total bandwidth of an optical circuit connection in the second set of optical circuit connections.

### Claim 31 (depends on 24)

The computer program product of claim 24, wherein the code further causes the apparatus to:
configure a network structure with a plurality of network PODs;
determine a plurality of spine switches based on at least the plurality of network PODs;
interconnect the plurality of network PODs via the plurality of spine switches; and
trigger the execution of the LLM task using the network structure.
