# US 8,570,865 B2 · Reducing power consumption in a fat-tree network > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 8,570,865 B2 - Document ID: US8570865B2 - Status: Granted - Kind: B2 - Application number: 13/026,309 - Filed: 2011-02-14 - Published: 2012-08-16 - Granted: 2013-10-29 - Earliest claimed: 2011-02-14 - Assignees: Mellanox Technologies, Ltd. - Applicants: Dror Goldenberg; Eitan Zahavi; Ori Rottenstreich - Inventors: Dror Goldenberg; Eitan Zahavi; Ori Rottenstreich - Web page: https://patentgrep.com/patent/US8570865B2 ## CPC - H04L 12/12: Arrangements for remote connection or disconnection of substations or of equipment thereof - H04L 5/0058: {Allocation criteria} - H04L 12/462: {LAN interconnection over a bridge based backbone} - H04L 43/0876: Network utilisation, e.g. volume of load or congestion level - H04L 45/243: using M+N parallel active paths - Y02D 30/50: in wire-line communication networks, e.g. low power modes or reduced link rate - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - Y02D: CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE ## Abstract A method for communication includes estimating a characteristic of communication traffic to be carried by a fat-tree network. Responsively to the estimated characteristic, a subset of the spine switches in the highest level of the network is selected, according to a predetermined selection order, to be active in carrying the communication traffic. In each of the levels of the spine switches below the highest level, the spine switches to be active are selected based on the selected spine switches in a next-higher level. The network is operated so as to convey the traffic between the leaf switches via the active spine switches, while the spine switches that are not selected remain inactive. ## Claims ### Claim 1 A method for communication, comprising: estimating a characteristic of communication traffic to be carried by a fat-tree network, which includes leaf switches interconnected by links with spine switches arranged in multiple levels, culminating in a highest level; responsively to the estimated characteristic, selecting, according to a predetermined selection order, a subset of the spine switches in the highest level to be active in carrying the communication traffic; in each of the levels of the spine switches below the highest level, selecting the spine switches to be active based on the selected spine switches in a next-higher level; and operating the network so as to convey the traffic between the leaf switches via the active spine switches, while the spine switches that are not selected remain inactive. ### Claim 2 (depends on 1) The method according to claim 1, wherein the selection order is determined so as to maximize a number of the spine switches in intermediate levels of the network that can be inactivated while still maintaining full connectivity between the leaf switches. ### Claim 3 (depends on 1) The method according to claim 1, wherein the network is a k-ary n-tree network, and wherein the selection order is determined such that the spine switches in the highest level that connect to a common spine switch in an intermediate level of the network are activated and deactivated in immediate succession. ### Claim 4 (depends on 3) The method according to claim 3, wherein the spine switches are identified by respective indices, and wherein the selection order is expressed as a mathematical progression over the indices. ### Claim 5 (depends on 1) The method according to claim 1, wherein estimating the characteristic of the communication traffic comprises scheduling computing jobs to be carried out by computing nodes connected to the leaf switches, and estimating characteristics of the communication traffic due to each of the computing jobs. ### Claim 6 (depends on 5) The method according to claim 5, wherein scheduling the jobs comprises determining the scheduling responsively to the selection order so as to reduce a power consumption of the network. ### Claim 7 (depends on 1) The method according to claim 1, wherein estimating the characteristic of the communication traffic comprises measuring a volume of the communication traffic carried by the network, and wherein selecting the subset of the spine switches comprises comparing the measured volume to a predetermined threshold in order to decide how many of the spine switches should be active. ### Claim 8 (depends on 1) The method according to claim 1, wherein estimating the characteristic of the communication traffic comprises detecting a change in the characteristic of the communication traffic, and wherein operating the network comprises changing an activity state of at least one of the spine switches responsively to the change. ### Claim 9 (depends on 8) The method according to claim 8, wherein operating the network comprises modifying a local routing of the communication traffic in a vicinity of the at least one of the spine switches responsively to the change. ### Claim 10 (depends on 8) The method according to claim 8, wherein operating the network comprises transmitting messages among the switches so as to modify a routing of the communication traffic with respect to the at least one of the spine switches prior to changing the activity state. ### Claim 11 A method for communication, comprising: receiving computing jobs that are to be executed by a system that includes a plurality of computing nodes connected to leaf switches of a fat-tree network, in which the leaf switches are interconnected by links with spine switches; scheduling the jobs to run on respective groups of the computing nodes, wherein the groups are selected so as to enable one or more of the spine switches to be deactivated during execution of at least one of the jobs; and executing the jobs on the respective groups of the computing nodes while deactivating the one or more of the spine switches in accordance with the scheduling. ### Claim 12 (depends on 11) The method according to claim 11, wherein scheduling the jobs comprises selecting the groups so that two or more of the jobs run concurrently on different groups of the computing nodes, and the spine switches interconnecting the different groups can be deactivated during the execution of the two or more of the jobs. ### Claim 13 (depends on 11) The method according to claim 11, wherein scheduling the jobs comprises estimating expected mutual traffic between pairs of the leaf switches due to the jobs, and positioning the leaf switches in the fat-tree network responsively to the mutual traffic. ### Claim 14 (depends on 13) The method according to claim 13, wherein positioning the leaf switches comprises assigning the pairs of the leaf switches that have high mutual traffic to adjacent positions in the fat-tree network. ### Claim 15 (depends on 11) The method according to claim 11, wherein scheduling the jobs comprises assigning the computing jobs to run during respective epochs, wherein the epochs and the groups of the computing nodes are chosen so as to reduce an overall power consumption of the network due to the spine switches that are active. ### Claim 16 (depends on 15) The method according to claim 15, wherein assigning the computing jobs comprises optimizing an objective function that depends on the overall power consumption and on delays incurred in running the jobs. ### Claim 17 Communication apparatus, comprising: a plurality of switches, comprising leaf switches and spine switches arranged in a fat-tree network, such that the leaf switches are interconnected by links with the spine switches arranged in multiple levels, culminating in a highest level; and a network manager, which is configured to select, according to a predetermined selection order and responsively to an estimated characteristic of communication traffic to be carried by the network, a subset of the spine switches in the highest level to be active in carrying the communication traffic, and in each of the levels of the spine switches below the highest level, to select the spine switches to be active based on the selected spine switches in a next-higher level, so as to cause the network to operate so as to convey the traffic between the leaf switches via the active spine switches, while the spine switches that are not selected remain inactive. ### Claim 18 (depends on 17) The apparatus according to claim 17, wherein the selection order is determined so as to maximize a number of the spine switches in intermediate levels of the network that can be inactivated while still maintaining full connectivity between the leaf switches. ### Claim 19 (depends on 17) The apparatus according to claim 17, wherein the network is a k-ary n-tree network, and wherein the selection order is determined such that the spine switches in the highest level that connect to a common spine switch in an intermediate level of the network are activated and deactivated in immediate succession. ### Claim 20 (depends on 19) The apparatus according to claim 19, wherein the spine switches are identified by respective indices, and wherein the selection order is expressed as a mathematical progression over the indices. ### Claim 21 (depends on 17) The apparatus according to claim 17, wherein the network manager is configured to schedule computing jobs to be carried out by computing nodes connected to the leaf switches, and to estimate characteristics of the communication traffic due to each of the computing jobs in order to select the spine switches that are to be active. ### Claim 22 (depends on 21) The apparatus according to claim 21, wherein the network manager is configured to schedule the jobs responsively to the selection order so as to reduce a power consumption of the network. ### Claim 23 (depends on 17) The apparatus according to claim 17, wherein the switches are configured to measure a volume of the communication traffic carried by the network, and wherein the network manager is configured to decide how many of the spine switches should be active by comparing the measured volume to a predetermined threshold. ### Claim 24 (depends on 17) The apparatus according to claim 17, wherein the network manager is configured to detect a change in the characteristic of the communication traffic, and to change an activity state of at least one of the spine switches responsively to the change. ### Claim 25 (depends on 24) The apparatus according to claim 24, wherein the switches are configured to modify a local routing of the communication traffic in a vicinity of the at least one of the spine switches responsively to the change. ### Claim 26 (depends on 24) The apparatus according to claim 24, wherein the switches are configured to transmit and receive messages so as to modify a routing of the communication traffic with respect to the at least one of the spine switches prior to changing the activity state. ### Claim 27 Communication apparatus, comprising: a computing system comprising a fat-tree network, which comprises leaf switches and spine switches interconnected by links, and a plurality of computing nodes connected to the leaf switches; and a network manager, which is configured to receive computing jobs that are to be executed by the computing system and to schedule the jobs to run on respective groups of the computing nodes, wherein the groups are selected so as to enable one or more of the spine switches to be deactivated during execution of at least one of the jobs, and to cause the jobs to be executed on the respective groups of the computing nodes as scheduled while deactivating the one or more of the spine switches. ### Claim 28 (depends on 27) The apparatus according to claim 27, wherein the network manager is configured to select the groups so that two or more of the jobs run concurrently on different groups of the computing nodes, and the spine switches interconnecting the different groups can be deactivated during the execution of the two or more of the jobs. ### Claim 29 (depends on 27) The apparatus according to claim 27, wherein the network manager is configured to estimate expected mutual traffic between pairs of the leaf switches due to the jobs, and to position the leaf switches in the fat-tree network responsively to the mutual traffic. ### Claim 30 (depends on 29) The apparatus according to claim 29, wherein the network manager is configured to assign the pairs of the leaf switches that have high mutual traffic to adjacent positions in the fat-tree network. ### Claim 31 (depends on 27) The apparatus according to claim 27, wherein the network manager is configured to assign the computing jobs to run during respective epochs, wherein the epochs and the groups of the computing nodes are chosen so as to minimize an overall energy consumption of the network due to the spine switches that are active. ### Claim 32 (depends on 31) The apparatus according to claim 31, wherein the network manager is configured to choose the epochs and the groups by optimizing an objective function that depends on the overall power consumption and on delays incurred in running the jobs.