# US 2021/0344782 A1 · Explicit notification of operative conditions along a network path > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0344782 A1 - Document ID: US20210344782A1 - Status: Published application - Kind: A1 - Application number: 17/198,292 - Filed: 2021-03-11 - Published: 2021-11-04 - Earliest claimed: 2020-05-03 - Assignees: Mellanox Technologies, Ltd. - Applicants: Mellanox Technologies, Ltd. - Inventors: Yuval Shpigelman; Idan Burstein; Aviv Kfir; Liron Mula; Niv Aibester; Gil Levy - Web page: https://patentgrep.com/patent/US20210344782A1 ## CPC - H04L 47/33: using forward notification - H04L 1/0002: {by adapting the transmission rate} - H04L 47/12: Avoiding congestion; Recovering from congestion - H04L 69/22: Parsing or analysis of headers - H04L 47/263: Rate modification at the source after receiving feedback - H04L 47/35: by embedding flow control information in regular packets, e.g. piggybacking - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract A network element includes circuitry and multiple ports. The multiple ports are configured to connect to a communication network. The circuitry is configured to receive via one of the ports a packet that originated from a source node and is destined to a destination node, the packet including a mark that is indicative of a cumulative state derived from at least bandwidth utilization conditions of output ports that were traversed by the packet along a path, from the source node up to the network element, to select a port for forwarding the packet toward the destination node, to update the mark of the packet based at least on a value of the mark in the received packet and on a local bandwidth utilization condition of the selected port, and to transmit the packet having the updated mark to the destination node via the selected port. ## Claims ### Claim 1 A network element, comprising: multiple ports configured to connect to a communication network; and circuitry configured to: receive via one of the ports a packet that originated from a source node and is destined to a destination node, the packet comprising a mark that is indicative of a cumulative state derived from at least bandwidth utilization conditions of output ports that were traversed by the packet along a path, from the source node up to the network element; select a port for forwarding the packet toward the destination node; update the mark of the packet based at least on a value of the mark in the received packet and on a local bandwidth utilization condition of the selected port; and transmit the packet having the updated mark to the destination node via the selected port. ### Claim 2 (depends on 1) The network element according to claim 1, wherein the cumulative state is derived from both the bandwidth utilization conditions and from congestion conditions of the output ports that were traversed by the packet along the path, and wherein the circuitry is configured to update the mark based on both the local bandwidth utilization condition and on a local congestion condition of the selected port. ### Claim 3 (depends on 2) The network element according to claim 2, wherein the mark is indicative of whether the path from the source node up to the network element is in a cumulative state selected from a list of cumulative states consisting at least: (i) a cumulative congestion state, in which one or more preceding ports along the path are congested, (ii) a cumulative normal state, in which all preceding ports along the path are uncongested and at least one of the preceding ports along the path is not underutilized, and (iii) a cumulative underutilization state, in which all preceding ports along the path are underutilized. ### Claim 4 (depends on 3) The network element according to claim 3, wherein the mark in the received packet is indicative of the cumulative congestion state, and wherein the circuitry is configured to retain the mark in the received packet in the cumulative congestion state, independently of the local congestion condition and of the local bandwidth utilization condition of the selected port. ### Claim 5 (depends on 3) The network element according to claim 3, wherein the circuitry is configured to, in response to detecting an imminent local congestion condition of the selected port, set the mark in the packet to the cumulative congestion state, independently of the local bandwidth utilization condition of the selected port. ### Claim 6 (depends on 3) The network element according to claim 3, wherein the mark in the received packet is indicative of the cumulative normal state, and wherein the circuitry is configured to, in response to detecting that the selected port is uncongested, retain the mark in the received packet in the cumulative normal state. ### Claim 7 (depends on 3) The network element according to claim 3, wherein the mark in the received packet is indicative of the cumulative underutilization state, and wherein the circuitry is configured to, in response to detecting that the selected port is uncongested, set the mark in the received packet based on the local bandwidth utilization condition of the selected port. ### Claim 8 (depends on 7) The network element according to claim 7, wherein the circuitry is configured to retain the mark in the received packet in the cumulative underutilization state, in response to detecting that the selected port is underutilized. ### Claim 9 (depends on 1) The network element according to claim 1, wherein the circuitry is configured to measure a bandwidth of data transmitted by the selected port, and to determine the local bandwidth utilization condition of the selected port, by applying a probabilistic function to the measured bandwidth. ### Claim 10 (depends on 1) The network element according to claim 1, wherein the mark reuses a field in a header of the packet that is used by a given protocol only for congestion notification. ### Claim 11 A method, comprising: in a network element comprising multiple ports that connect to a communication network, receiving via one of the ports a packet that originated from a source node and is destined to a destination node, the packet comprising a mark that is indicative of a cumulative state derived from at least bandwidth utilization conditions of output ports that were traversed by the packet along a path, from the source node up to the network element; selecting a port for forwarding the packet toward the destination node; updating the mark of the packet based at least on a value of the mark in the received packet and on a local bandwidth utilization condition of the selected port; and transmitting the packet having the updated mark to the destination node via the selected port. ### Claim 12 (depends on 11) The method according to claim 11, wherein the cumulative state is derived from both the bandwidth utilization conditions and from congestion conditions of the output ports that were traversed by the packet along the path, and wherein updating the mark comprises updating the mark based on both the local bandwidth utilization condition and on a local congestion condition of the selected port. ### Claim 13 (depends on 12) The method according to claim 12, wherein the mark is indicative of whether the path from the source node up to the network element is in a cumulative state selected from a list of cumulative states consisting at least: (i) a cumulative congestion state, in which one or more preceding ports along the path are congested, (ii) a cumulative normal state, in which all preceding ports along the path are uncongested and at least one of the preceding ports along the path is not underutilized, and (iii) a cumulative underutilization state, in which all preceding ports along the path are underutilized. ### Claim 14 (depends on 13) The method according to claim 13, wherein the mark in the received packet is indicative of the cumulative congestion state, and wherein updating the mark comprises retaining the mark in the received packet in the cumulative congestion state, independently of the local congestion condition and of the local bandwidth utilization condition of the selected port. ### Claim 15 (depends on 13) The method according to claim 13, wherein updating the mark comprises, in response to detecting an imminent local congestion condition of the selected port, setting the mark in the packet to the cumulative congestion state, independently of the local bandwidth utilization condition of the selected port. ### Claim 16 (depends on 13) The method according to claim 13, wherein the mark in the received packet is indicative of the cumulative normal state, and wherein updating the mark comprises, in response to detecting that the selected port is uncongested, retaining the mark in the received packet in the cumulative normal state. ### Claim 17 (depends on 13) The method according to claim 13, wherein the mark in the received packet is indicative of the cumulative underutilization state, and wherein updating the mark comprises, in response to detecting that the selected port is uncongested, setting the mark in the received packet based on the local bandwidth utilization condition of the selected port. ### Claim 18 (depends on 17) The method according to claim 17, wherein updating the mark comprises retaining the mark in the received packet in the cumulative underutilization state, in response to detecting that the selected port is underutilized. ### Claim 19 (depends on 11) The method according to claim 11, and comprising measuring a bandwidth of data transmitted by the selected port, and determining the local bandwidth utilization condition of the selected port, by applying a probabilistic function to the measured bandwidth. ### Claim 20 (depends on 11) The method according to claim 11, wherein the mark reuses a field in a header of the packet that is used by a given protocol only for congestion notification. ### Claim 21 A network node comprising: a network adapter coupled to a communication network; and a host processor coupled to the network adapter, configured to: receive from a remote node that is coupled to the communication network via the network adapter, a packet comprising a mark that is indicative of a cumulative state derived from at least bandwidth utilization conditions of output ports that were traversed by the packet along a path, from the network node to the remote node; and adjust a rate of data transmission by the host processor into the communication network based on the mark in the received packet. ### Claim 22 (depends on 21) The network node according to claim 21, wherein the cumulative state is derived from both the bandwidth utilization conditions and from congestion conditions of the output ports that were traversed by the packet along the path. ### Claim 23 (depends on 22) The network node according to claim 22, wherein the mark is indicative of whether the path from the network node to the remote node is in a cumulative state selected from a list of cumulative states consisting at least: (i) a cumulative congestion state, in which one or more ports along the path are congested, (ii) a cumulative normal state, in which all the ports along the path are uncongested and at least one of the ports along the path is not underutilized, and (iii) a cumulative underutilization state, in which all the ports along the path are underutilized. ### Claim 24 (depends on 23) The network node according to claim 23, wherein the host processor is configured to, responsively to the mark being indicative of the cumulative underutilization state, increase the rate of data transmission into the communication network. ### Claim 25 (depends on 23) The network node according to claim 23, wherein the host processor is configured to, responsively to the mark being indicative of the cumulative normal state, increase the rate of data transmission into the communication network at a first pace, and responsively to the mark being indicative of the cumulative underutilization state, increase the rate of data transmission to the communication network at a second pace that is faster than the first pace. ### Claim 26 A method comprising: in a network node, receiving by a host processor, a packet sent from a remote node over a communication network, the packet comprising a mark that is indicative of a cumulative state derived from bandwidth utilization conditions of output ports that were traversed by the packet along a path, from the network node to the remote node; and adjusting, by the host processor, a rate of data transmission into the communication network based on the mark in the received packet. ### Claim 27 (depends on 26) The method according to claim 26, wherein the cumulative state is derived from both the bandwidth utilization conditions and from congestion conditions of the output ports that were traversed by the packet along the path. ### Claim 28 (depends on 27) The method according to claim 27, wherein the mark is indicative of whether the path from the network node to the remote node is in a cumulative state selected from a list of cumulative states consisting at least: (i) a cumulative congestion state, in which one or more ports along the path are congested, (ii) a cumulative normal state, in which all the ports along the path are uncongested and at least one of the ports along the path is not underutilized, and (iii) a cumulative underutilization state, in which all the ports along the path are underutilized. ### Claim 29 (depends on 28) The method according to claim 28, wherein adjusting the rate of data transmission comprises, responsively to the mark being indicative of the cumulative underutilization state, increasing the rate of data transmission into the communication network. ### Claim 30 (depends on 28) The method according to claim 28, wherein adjusting the rate of data transmission comprises, responsively to the mark being indicative of the cumulative normal state, increasing the rate of data transmission into the communication network at a first pace, and responsively to the mark being indicative of the cumulative underutilization state, increasing the rate of data transmission to the communication network at a second pace that is faster than the first pace.