# US 2019/0173793 A1 · Method and apparatus for low latency data center network > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0173793 A1 - Document ID: US20190173793A1 - Status: Published application - Kind: A1 - Application number: 15/829,319 - Filed: 2017-12-01 - Published: 2019-06-06 - Earliest claimed: 2017-12-01 - Assignees: Futurewei Technologies, Inc. - Applicants: Futurewei Technologies, Inc. - Inventors: Fangping Liu; Zhenjiang Li; Serhat Nazim Avci - Web page: https://patentgrep.com/patent/US20190173793A1 ## CPC - H04L 47/25: with rate being modified by the source upon detecting a change of network conditions - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract Methods and apparatus for determining and adjusting traffic rate limits to reduce queuing delays and latency in a data center network. In various embodiments, a network controller receives network topology information and traffic data relating to flows of the network. The traffic data includes traffic matrix information, flow path information, and pause frame distribution information. Based on this information, the network controller identifies server network node interfaces that are the source of a pause frame. Flows associated with such network node interfaces are also identified and ranked based on traffic volume. For ranked flows meeting predetermined criteria, reduced traffic rate limits are calculated and sent to associated server network node interfaces. In some embodiments, the reduced traffic rate limits are incrementally recovered when normal traffic resumes on a paused link. In further embodiments, traffic rate limits are also adjusted for flows of highly utilized and underutilized network node interfaces. ## Claims ### Claim 1 A computer-implemented method for reducing latency in a network, the method comprising: receiving network topology information relating to a plurality of servers of a network; receiving traffic data relating to flows of the network, the traffic data including traffic matrix information, flow path information, and pause frame distribution information; identifying, based on the network topology information and the pause frame distribution information, one or more network node interfaces of the plurality of servers that are a source of a pause frame; identifying, based on the traffic data, flows of the one or more network node interfaces; determining, based on the traffic data and predetermined criteria, a reduced traffic rate limit for each of a plurality of the identified flows; and sending the reduced traffic rate limits for application to the one or more network node interfaces. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, wherein the predetermined criteria comprises: ranking the flows of an identified network node interface based on respective flow traffic volume values of the flows; and identifying one or more of the ranked flows having a cumulative traffic volume that constitutes a predetermined portion of a total flow traffic volume of the identified network node interface. ### Claim 3 (depends on 2) The computer-implemented method of claim 2, wherein determining a reduced traffic rate limit for each of the plurality of identified flows comprises: determining, for each of the identified ranked flows, a current traffic rate limit; and decreasing the current traffic rate limits by a percentage basis. ### Claim 4 (depends on 1) The computer-implemented method of claim 1 further comprising: receiving updated pause frame distribution information; identifying, based on the network topology information and the updated pause frame distribution information, a network node interface, of the one or more network node interfaces, that is no longer a source of a pause frame based on the reduced traffic rate limits; determining adjusted traffic rate limits by incrementally increasing the reduced traffic rate limits for the identified flows of the network node interface; and sending the adjusted traffic rate limits for application to the network node interface. ### Claim 5 (depends on 4) The computer-implemented method of claim 4 further comprising: receiving further updated pause frame distribution information; when the further updated pause frame distribution information indicates that the network node interface is a source of a pause frame based on the adjusted traffic rate limits: determining further adjusted traffic rate limits by incrementally decreasing the reduced traffic rate limits for the identified flows of the network node interface; when the further updated pause frame distribution information indicates that the network node interface is not a source of a pause frame based on the adjusted traffic rate limits: determining further adjusted traffic rate limits by incrementally increasing the reduced traffic rate limits for the identified flows of the network node interface; and sending the further adjusted traffic rate limits for application to the network node interface. ### Claim 6 (depends on 1) The computer-implemented method of claim 1 further comprising: identifying, based on the network topology information and the traffic data, additional network node interfaces of the plurality of servers having a utilization rate greater than a predetermined threshold; identifying, based on the traffic data, additional flows of the additional network node interfaces; determining additional reduced traffic rate limits for one or more of the additional flows of the additional node interfaces; and sending the additional reduced traffic rate limits for application to the additional network node interfaces. ### Claim 7 (depends on 1) The computer-implemented method of claim 1 further comprising: identifying, based on the network topology information and the traffic data, additional network node interfaces of the plurality of servers having a utilization rate less than a predetermined threshold; identifying, based on the traffic data, additional flows of the additional network node interfaces, wherein each network node interface of an additional flow is an additional network node interface; determining increased traffic rate limits for the additional flows of the additional network node interfaces; and sending the increased traffic rate limits for application to the additional network node interfaces. ### Claim 8 (depends on 1) The computer-implemented method of claim 1, further comprising: identifying, based on the traffic data, initial flows of the plurality of servers; determining, based on the traffic data, an initial traffic rate limit for each of the initial flows; and sending the initial rate limits for application to the plurality of servers. ### Claim 9 (depends on 1) The computer-implemented method of claim 1, wherein the network is a data center network. ### Claim 10 (depends on 1) The computer-implemented method of claim 1, wherein the method is repeated on a periodic basis. ### Claim 11 A device comprising: a memory storage comprising instructions; at least one network interface; and one or more processors in communication with the memory storage, wherein the one or more processors execute the instructions to: receive, via the at least one network interface, network topology information relating to a plurality of servers of a network; receive, via the at least one network interface, traffic data relating to flows of the network, the traffic data including traffic matrix information, flow path information, and pause frame distribution information; identify, based on the network topology information and the pause frame distribution information, one or more network node interfaces of the plurality of servers that are a source of a pause frame; identify, based on the traffic data, flows of the one or more network node interfaces; determine, based on the traffic data and predetermined criteria, a reduced traffic rate limit for each of a plurality of the identified flows; and send, via the at least one network interface, the reduced traffic rate limits for application to the one or more network node interfaces. ### Claim 12 (depends on 11) The device of claim 11, wherein the predetermined criteria comprises: ranking the flows of an identified network node interface based on respective flow traffic volume; and identifying one or more of the ranked flows having a cumulative traffic volume that constitutes a predetermined portion of a total flow traffic volume of the identified network node interface. ### Claim 13 (depends on 11) The device of claim 11, wherein the one or more processors further execute the instructions to: receive, via the at least one network interface, updated pause frame distribution information; identify, based on the network topology information and the updated pause frame distribution information, a network node interface, of the one or more network node interfaces, that is no longer a source of a pause frame based on the reduced traffic rate limits; determine adjusted traffic rate limits by incrementally increasing the reduced traffic rate limits for the identified flows of the network node interface; and send, via the at least one network interface, the adjusted traffic rate limits for application to the network node interface. ### Claim 14 (depends on 13) The device of claim 13, wherein the one or more processors further execute the instructions to: receive, via the at least one network interface, further updated pause frame distribution information; when the further updated pause frame distribution information indicates that the network node interface is a source of a pause frame based on the adjusted traffic rate limits: determine further adjusted traffic rate limits by incrementally decreasing the reduced traffic rate limits for the identified flows of the network node interface; when the further updated pause frame distribution information indicates that the network node interface is not a source of a pause frame based on the adjusted traffic rate limits: determine further adjusted traffic rate limits by incrementally increasing the reduced traffic rate limits for the identified flows of the network node interface; and send, via the at least one network interface, the further adjusted traffic rate limits for application to the network node interface for application to the additional network node interfaces. ### Claim 15 (depends on 11) The device of claim 11, wherein the one or more processors further execute the instructions to: identify, based on the network topology information and the traffic data, additional network node interfaces of the plurality of servers having a utilization rate greater than a first predetermined threshold; identify, based on the traffic data, additional flows of the additional network node interfaces; determine additional reduced traffic rate limits for one or more of the additional flows of the additional network node interfaces; and send, via the at least one network interface, the additional reduced traffic rate limits for application to the additional network node interfaces. ### Claim 16 (depends on 11) The device of claim 11, wherein the one or more processors further execute the instructions to: identify, based on the network topology information and the traffic data, additional network node interfaces of the plurality of servers having a utilization rate less than a predetermined threshold; identify, based on the traffic data, additional flows of the additional network node interfaces, wherein each network node interface of an additional flow is an additional network node interface; determine increased traffic rate limits for the additional flows of the additional network node interfaces; and send, via the at least one network interface, the increased traffic rate limits for application to the additional network node interfaces. ### Claim 17 (depends on 11) The device of claim 11, wherein the one or more processors execute the instructions on a periodic basis. ### Claim 18 (depends on 11) The device of claim 11, wherein the one or more processors further execute the instructions to: identify, based on the traffic data, initial flows of the plurality of servers; determine, based on the traffic data, an initial traffic rate limit for each of the initial flows; and send, via the at least one network interface, the initial rate limits for application by the plurality of servers. ### Claim 19 A computer-implemented method for reducing latency in a network, the method comprising: receiving network topology information relating to a plurality of servers of a network; receiving traffic data relating to flows of the network, the traffic data including traffic matrix information, flow path information, and pause frame distribution information; identifying, based on the network topology information and the pause frame distribution information, one or more network node interfaces of the plurality of servers that are a source of a pause frame; identifying, based on the traffic data, first flows of the one or more network node interfaces; determining, based on the traffic data and predetermined criteria, a reduced traffic rate limit for each of a plurality of the first flows; sending the reduced traffic rate limits for application to the one or more network node interfaces; identifying, based on the network topology information and the traffic data, first additional network node interfaces of the plurality of servers having a utilization rate greater than a first predetermined threshold; identifying, based on the traffic data, first additional flows of the first additional network node interfaces; determining additional reduced traffic rate limits for one or more of the first additional flows of the first additional network node interfaces; sending the additional reduced traffic rate limits for application to the first additional network node interfaces; identifying, based on the network topology information and the traffic data, second additional network node interfaces of the plurality of servers having a utilization rate less than a second predetermined threshold; identifying, based on the traffic data, second additional flows, wherein each network node interface of a second additional flow is a second additional network node interface; determining increased traffic rate limits for the second additional flows of the second additional network node interfaces; and sending the increased traffic rate limits for application to the second additional network node interfaces. ### Claim 20 (depends on 19) The computer-implemented method of claim 19, further comprising: receiving updated pause frame distribution information; identifying, based on the network topology information and the updated pause frame distribution information, a network node interface, of the one or more network node interfaces, that is no longer a source of a pause frame based on the reduced traffic rate limits; determining adjusted traffic rate limits by incrementally increasing the reduced traffic rate limits for the first flows of the network node interface; and sending the adjusted traffic rate limits for application to the network node interface.