# US 11,115,341 B1 · Low latency flow control in data centers > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,115,341 B1 - Document ID: US11115341B1 - Status: Granted - Kind: B1 - Application number: 16/578,344 - Filed: 2019-09-22 - Granted: 2021-09-07 - Earliest claimed: 2018-10-18 - Assignees: Xsight Labs Ltd. - Applicants: Xsight Labs Ltd. - Inventors: Guy Koren; Gal Malach; Erez Shaizaf - Web page: https://patentgrep.com/patent/US11115341B1 ## CPC - H04L 47/12: Avoiding congestion; Recovering from congestion - H04L 45/122: by minimising distances, e.g. by selecting a route with minimum of number of hops - H04L 47/15: {in relation to multipoint traffic (arrangements for broadcast or multicast in data networks H04L12/18)} - H04L 47/22: Traffic shaping - H04L 47/2483: involving identification of individual flows - H04L 47/30: in combination with information about buffer occupancy at either end or at transit nodes - H04L 47/50: Queue scheduling - H04L 47/6255: {queue load conditions, e.g. longest queue first} - H04L 47/6295: using multiple queues, one for each individual QoS, connection, flow or priority - H04L 49/1569: {Clos switching fabrics} - H04L 49/3045: {Virtual queuing} - H04L 47/125: by balancing the load, e.g. traffic engineering - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract A system for managing traffic between servers, the system may include first tier switches that are coupled to the servers; second tier switches that are coupled to the first tier switches and to third tier switches; and controllers. Wherein each first tier switch comprises first queues. Wherein each second tier switch comprises second queues. The controllers are configured to control a traffic between the first tier switches and the second tier switches attributed to the traffic between the servers, (a) on, at least, a queue granularity; (b) while controlling some first queues to provide buffer extension to some second queues, and (c) while controlling some second queues to provide buffer extension to some first queues. ## Claims ### Claim 1 A system for managing traffic between servers, the system comprises: first tier switches that are coupled to the servers; second tier switches that are coupled to the first tier switches and to third tier switches; and controllers; wherein each first tier switch comprises first queues; wherein each second tier switch comprises second queues; wherein the controllers are configured to control a traffic between the first tier switches and the second tier switches attributed to the traffic between the servers, (a) on, at least, a queue granularity; (b) while controlling some first queues to provide buffer extension to some second queues, and (c) while controlling some second queues to provide buffer extension to some first queues; wherein the first tier switches and the second tier switches are positioned in multiple pods; and wherein at least one of the controllers is configured to (a) prevent traffic between servers coupled to a same first tier switch from reaching any second tier switch, (b) prevent traffic between servers coupled to different first tier switches of a same pod from reaching any third tier switches, and (c) pass traffic between servers coupled to different pods through one or more third tier switches. ### Claim 2 (depends on 1) The system according to claim 1 wherein at least one of the controllers is configured to control the traffic between the first tier switches and the second tier switches on a queue granularity and on at least one additional granularity that is coarser than the queue granularity. ### Claim 3 (depends on 2) The system according to claim 2 wherein the at least one additional granularity is a group of queues granularity. ### Claim 4 (depends on 2) The system according to claim 2 wherein the at least one additional granularity is a switch granularity. ### Claim 5 (depends on 1) The system according to claim 1 wherein at least one of the controllers is configured to control traffic from a source server to a destination server while enforcing a minimal hop approach. ### Claim 6 (depends on 1) The system according to claim 1 wherein at least one of the controllers is configured to control traffic from a source server to a destination server, when the source server and the destination server are coupled to a same first tier switch, by conveying the traffic through a first tier switch and without sending the traffic to a second tier switch. ### Claim 7 (depends on 1) The system according to claim 1 wherein at least one of the controllers is configured to control traffic from a source server that is coupled to a first tier source switch to a destination server that is coupled to a first tier destination switch, through second buffer extension queues of certain second tier switch, and through a first queue of a destination first tier switch, when the source first tier switch and the destination first tier switch differ from each other but belong to the same pod of the multiple pods, by conveying the traffic through a first input queue of a source first tier switch. ### Claim 8 (depends on 1) The system according to claim 1 wherein at least one of the controllers is configured to control traffic from a source server that is coupled to a first tier source switch to a destination server that is coupled to a first tier destination switch, by conveying the traffic through a first queue of the source first tier switch, through a second buffer extension queue of certain second tier switch, towards a third tier switch, from another third tier switch, through another second tier switch and through a destination first tier switch. ### Claim 9 A system for managing traffic between servers, the system comprises: first tier switches that are coupled to the servers; second tier switches that are coupled to the first tier switches and to third tier switches; and controllers; wherein each first tier switch comprises first queues; wherein each second tier switch comprises second queues; wherein the controllers are configured to control a traffic between the first tier switches and the second tier switches attributed to the traffic between the servers, (a) on, at least, a queue granularity; (b) while controlling some first queues to provide buffer extension to some second queues, and (c) while controlling some second queues to provide buffer extension to some first queues; and wherein at least one of the controllers is configured to control traffic between a certain first queue and certain second queues that provide buffer extension to the first queue by (a) monitoring a status of the certain queue, (2) detecting a potential congestion when a fullness level of the first queue exceeds a predefined threshold but is lower that a maximal fullness level, (c) sending a potential congestion alert, and (d) initiating a congestion prevention step related to the certain second queues. ### Claim 10 (depends on 9) The system according to claim 9 wherein the at least one of the controller is configured to detect the potential congestion, generate the potential congestion alert and to initiate the congestion prevention step within less that a ten microseconds. ### Claim 11 A method for managing traffic between servers, the method comprises: controlling, by controllers of a system, traffic between servers, wherein the system comprises (i) first tier switches that are coupled to the servers; (ii) second tier switches that are coupled to the first tier switches and to third tier switches; wherein each first tier switch comprises first queues; wherein each second tier switch comprises second queues; wherein the controlling of the traffic comprises controlling a traffic between the first tier switches and the second tier switches that is attributed to the traffic between the servers, (a) on, at least, a queue granularity; (b) while controlling some first queues to provide buffer extension to some second queues, and (c) while controlling some second queues to provide buffer extension to some first queues; wherein the first tier switches and the second tier switches are positioned in multiple pods; and wherein the method further comprises comprising (a) preventing traffic between servers coupled to a same first tier switch from reaching any second tier switch, (b) preventing traffic between servers coupled to different first tier switches of a same pod from reaching any third tier switches, and (c) passing traffic between servers coupled to different pods through one or more third tier switches. ### Claim 12 (depends on 11) The method according to claim 11 comprising controlling, by at least one of the controllers, the traffic between the first tier switches and the second tier switches on a queue granularity and on at least one additional granularity that is coarser than the queue granularity. ### Claim 13 (depends on 12) The method according to claim 12 wherein the at least one additional granularity is a group of queues granularity. ### Claim 14 (depends on 12) The method according to claim 12 wherein the at least one additional granularity is a switch granularity. ### Claim 15 (depends on 11) The method according to claim 11 comprising controlling, by at least one of the controllers, traffic from a source server to a destination server while enforcing a minimal hop approach. ### Claim 16 (depends on 11) The method according to claim 11 wherein comprising controlling, by at least one of the controllers, traffic from a source server to a destination server, when the source server and the destination server are coupled to a same first tier switch, by conveying the traffic through a first tier switch and without sending the traffic to a second tier switch. ### Claim 17 (depends on 11) The method according to claim 11 controlling, by at least one of the controllers, traffic from a source server that is coupled to a first tier source switch to a destination server that is coupled to a first tier destination switch, through second buffer extension queues of certain second tier switch, and through a first queue of a destination first tier switch, when the source first tier switch and the destination first tier switch differ from each other but belong to the same pod of the multiple pods, by conveying the traffic through a first input queue of a source first tier switch. ### Claim 18 (depends on 11) The method according to claim 11 controlling, by at least one of the controllers, traffic from a source server that is coupled to a first tier source switch to a destination server that is coupled to a first tier destination switch, by conveying the traffic through a first queue of the source first tier switch, through a second buffer extension queue of certain second tier switch, towards a third tier switch, from another third tier switch, through another second tier switch and through a destination first tier switch. ### Claim 19 A method for managing traffic between servers, the method comprises: controlling, by controllers of a system, traffic between servers, wherein the system comprises (i) first tier switches that are coupled to the servers; (ii) second tier switches that are coupled to the first tier switches and to third tier switches; wherein each first tier switch comprises first queues; wherein each second tier switch comprises second queues; wherein the controlling of the traffic comprises controlling a traffic between the first tier switches and the second tier switches that is attributed to the traffic between the servers, (a) on, at least, a queue granularity; (b) while controlling some first queues to provide buffer extension to some second queues, and (c) while controlling some second queues to provide buffer extension to some first queues; and controlling, by at least one of the controllers, traffic between a certain first queue and certain second queues that provide buffer extension to the first queue by (a) monitoring a status of the certain queue, (2) detecting a potential congestion when a fullness level of the first queue exceeds a predefined threshold but is lower that a maximal fullness level, (c) sending a potential congestion alert, and (d) initiating a congestion prevention step related to the certain second queues. ### Claim 20 (depends on 19) The method according to claim 19 wherein the detecting of the potential congestion, the generating of the potential congestion alert and the initiating of the congestion prevention is executed within less that a ten microseconds. ### Claim 21 A non-transitory computer readable medium that stores instructions for managing traffic between servers, the instructions, once executed by a system cause the system to execute the steps of: controlling, by controllers of the system, traffic between servers, wherein the system comprises (i) first tier switches that are coupled to the servers; (ii) second tier switches that are coupled to the first tier switches and to third tier switches; wherein each first tier switch comprises first queues; wherein each second tier switch comprises second queues; and wherein the controlling of the traffic comprises controlling a traffic between the first tier switches and the second tier switches that is attributed to the traffic between the servers, (a) on, at least, a queue granularity; (b) while controlling some first queues to provide buffer extension to some second queues, and (c) while controlling some second queues to provide buffer extension to some first queues; and controlling, by at least one of the controllers, traffic between a certain first queue and certain second queues that provide buffer extension to the first queue by (a) monitoring a status of the certain queue, (2) detecting a potential congestion when a fullness level of the first queue exceeds a predefined threshold but is lower that a maximal fullness level, (c) sending a potential congestion alert, and (d) initiating a congestion prevention step related to the certain second queues.