# US 12,764,061 B2 · Using control plane circuits of managed forwarding elements to configure endpoint processing units to use the network fabric connecting the endpoint processing units > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,764,061 B2 - Document ID: US12764061B2 - Status: Granted - Kind: B2 - Application number: 19/227,464 - Filed: 2025-06-04 - Published: 2026-03-26 - Granted: 2026-10-13 - Earliest claimed: 2024-09-21 - Assignees: Delos Data Inc. - Applicants: Delos Data Inc. - Inventors: Daniel P. Daly; Edward V. E. Doe; Alain J. E. Gravel - Web page: https://patentgrep.com/patent/US12764061B2 ## CPC - H04L 45/24: Multipath - H04L 45/123: {Evaluation of link metrics (techniques for monitoring network metrics H04L43/08)} - H04L 47/33: using forward notification - H04L 67/10: in which an application is distributed across nodes in the network (software deployment G06F8/60; multiprogramming arrangements G06F9/46) - H04L 67/1001: for accessing one among a plurality of replicated servers - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract Some embodiments provide a method of managing communication between endpoint processing units (EPUs) connected through a network having multiple forwarding elements, each of which has a data plane circuit to forward data messages exchanged between EPUs and a control plane circuit to configure the data plane circuit. The method assigns, for each particular EPU, one particular CP circuit of one forwarding element to operate as a leader CP circuit to configure the network interface of the particular EPU to forward data messages through the network. The method distributes configuration data to the CP circuits of the forwarding elements. Each particular CP circuit uses its received configuration data to instruct the network interfaces of each particular EPU for which the CP circuit is a leader to perform the desired forwarding operations to forward results of computations of the particular EPUs through the network. ## Claims ### Claim 1 A method of managing communication between endpoint processing units (EPUs) connected through a network comprising a plurality of forwarding elements each of which has a data plane circuit to forward data messages exchanged between EPUs and a control plane (CP) circuit to configure the data plane circuit, the method comprising: assigning, for each particular EPU, one particular CP circuit of one forwarding element to operate as a leader CP circuit to configure a network interface of the particular EPU to forward data messages through the network; and distributing configuration data to the CP circuits of the forwarding elements, each particular CP circuit using its received configuration data to instruct the network interfaces of each particular EPU for which the CP circuit is a leader to perform desired forwarding operations to forward results of computations of the particular EPUs through the network. ### Claim 2 (depends on 1) The method of claim 1, wherein the EPUs are graphics processing units (GPUs). ### Claim 3 (depends on 1) The method of claim 1, wherein the EPUs comprise at least one of graphics processing units (GPUs), tensor processing units (TPUs) and central processing units (CPUs). ### Claim 4 (depends on 1) The method of claim 1, wherein based on the distributed configuration data, each leader CP circuit for each particular EPU provides forwarding records to the network interface of the particular EPU to use in forwarding different results to destinations in the network that should receive the results. ### Claim 5 (depends on 4) The method of claim 4, wherein the forwarding records are part of the distributed configuration data. ### Claim 6 (depends on 4) The method of claim 4, wherein each CP circuit generates at least one forwarding record from the distributed configuration data. ### Claim 7 (depends on 1) The method of claim 1, wherein based on the distributed configuration data, each leader CP circuit for each particular EPU provides scheduling parameters to the network interface of the particular EPU to use to schedule timing or rate for forwarding different results to destinations in the network that should receive the results. ### Claim 8 (depends on 7) The method of claim 7, wherein the scheduling parameters are part of the distributed configuration data. ### Claim 9 (depends on 7) The method of claim 7, wherein each CP circuit generates at least a subset of the scheduling parameters from the distributed configuration data. ### Claim 10 (depends on 7) The method of claim 7, wherein the leader CP circuit provides different scheduling parameters for different results in a first set of results as part of proactively prescheduling the forwarding of the different results in the first set before the EPU performs different computations in a first set of computations that produce the different results in the first set. ### Claim 11 (depends on 10) The method of claim 10, wherein proactively prescheduling the different results in the first set comprises proactively providing each scheduling parameter for each result in the first set to a particular EPU before the particular EPU has been assigned the computation in the first set of computations that produces the result. ### Claim 12 (depends on 10) The method of claim 10, wherein each CP circuit in at least a set of CP circuits reactively provides scheduling parameters to network interfaces of different EPUs when (i) an EPU does not have proactively provided scheduling parameter for a result in a second set of unscheduled results and (ii) the EPU sends an in-band data message through the network to obtain the scheduling parameter, each in-band data message forwarded to a CP circuit to process to supply the reactively provided scheduling parameter. ### Claim 13 (depends on 12) The method of claim 12, wherein the CP circuit that supplies the reactively provided scheduling parameter for a result that has to be sent from a source EPU to a destination EPU is a CP circuit in a last hop forwarding element in the path between the source and destination EPUs. ### Claim 14 (depends on 12) The method of claim 12, wherein each CP circuit in the set of CP circuits supplies to an EPU network interface a reactively provided scheduling parameter for a result through an in-band data message that is carried to the EPU network interface through a set of one or more data plane circuits. ### Claim 15 (depends on 1) The method of claim 1, wherein instruction from each CP circuit are forwarded in-band to each EPU network interface through one or more data plane circuits of one or more of the forwarding elements. ### Claim 16 (depends on 1) The method of claim 1, wherein the leader CP circuit for each particular EPU is a CP circuit of a forwarding element that is one-hop away in the network from the particular EPU as the forwarding element is connected with the particular EPU through a physical link. ### Claim 17 A non-transitory machine readable medium storing program that when executed by at least one processor manages communication between graphics processing units (GPUs) connected through a network comprising a plurality of forwarding elements each of which has a data plane circuit to forward data messages exchanged between GPUs and a control plane (CP) circuit to configure the data plane circuit, the program comprising sets of instructions for: assigning, for each particular GPU, one particular CP circuit of one forwarding element to operate as a leader CP circuit to configure a network interface of the particular GPU to forward data messages through the network; and distributing configuration data to the CP circuits of the forwarding elements, each particular CP circuit using its received configuration data to instruct the network interfaces of each particular GPU for which the CP circuit is a leader to perform desired forwarding operations to forward results of computations of the particular GPUs through the network. ### Claim 18 (depends on 17) The non-transitory machine readable medium of claim 17, wherein based on the distributed configuration data, each leader CP circuit for each particular GPU provides forwarding records to the network interface of the particular GPU to use in forwarding different results to destinations in the network that should receive the results. ### Claim 19 (depends on 18) The non-transitory machine readable medium of claim 18, wherein the forwarding records are part of the distributed configuration data. ### Claim 20 (depends on 18) The non-transitory machine readable medium of claim 18, wherein each CP circuit generates at least one forwarding record from the distributed configuration data.