US 12,764,025 B2 · Topologies for scale-out fabrics

Abstract

Recently, there has been a dramatic increase in the development and use of machine learning and artificial intelligence (ML/AI) applications. The exponentially growing complexity of ML/AI models and their voracious data needs have created needs for large but well-designed data centers—even the network topology needs to be critically considered. Presented herein are embodiments of topologies for data center system that comprise both a high bandwidth scale-up fabric and a scale-out fabric. In one or more embodiments, the scale-out fabric may comprise as many rails as allowed by the radix of the switch (i.e., 36 rails for 64-port switches and 72 rails for 128-port switches), and in one or more embodiments, the rails preferably span a minimum number of tiers as possible and reserve the upper tier for the rail crossing function. If the last tier is used for the rail crossing function, it may be oversubscribed.

Claims20

Dependent claims

A method for configuring a network topology comprising:

given a set of one or more railpods, in which each railpod comprises a plurality of racks and each rack comprises:
a set of compute nodes;
n number of ports;
and
a first network that interconnects processors of the compute nodes;
connecting each rack via a number of connections (c) to a set of leaf switches (l) in which:
c×l equals an integer divisor of the number of ports (n) of a rack;
each leaf switch forms a rail;
and
the number of ports of a leaf switch of the set of leaf switches is not an integer multiple of the number of ports (n) of a rack;
and
responsive to extending the rails to a second tier comprising spine switches:
connecting the leaf switches to sets of two or more spine switches, in which:
each set of two or more spine switches corresponds to a rail;
and
within a rail, each leaf switch is connected to the spine switches with a number of connections dependent upon a cluster scale;
and
connecting each of the sets of spine switches to a corresponding row of superspine switches to facilitate data crossing rails.

15 more elements

A network system comprising:

a set of processors, in which each processor comprises two corresponding interfaces-one interface for connecting to a scale-up network and one interface for connecting to a scale-out network;
and
a scale-out network using radix-64 switches or a scale-out network using radix-128 switches,
wherein, when the scale-out network uses radix-64 switches, the scale-out network comprises:
the set of processors grouped in a railpod comprising 16 groups of 72 processors;
36 leaf switches for connecting each group of 72 processors to the 36 leaf switches via two links, in which each leaf switch is a network rail;
groups of 36 spine switches for connecting to the 36 leaf switches, in which each group of 36 spine switches is a network rail corresponding to the network rail of its connected leaf switch;
and
groups of 32 superspine switches for connecting to the groups of 36 spine switches, in which each group of 32 superspine switches is configured to connect with a corresponding group of 36 spine switches;
and
wherein, when the scale-out network uses radix-128 switches, the scale-out network comprises:
the set of processors grouped in a railpod comprising 64 groups of 72 processors;
72 leaf switches for connecting each group of 72 processors to the 72 leaf switches via one link, in which each leaf switch is a network rail;
groups of 72 spine switches for connecting to the 72 leaf switches, in which each group of 72 spine switches is a network rail corresponding to the network rail of its connected leaf switch;
and
groups of 64 superspine switches for connecting to the groups of 72 spine switches, in which each group of 64 superspine switches is configured to connect with a corresponding group of 72 spine switches.

13 more elements

A network system comprising:

a set of processors, in which each processor comprises two corresponding interfaces-one interface for connecting to a scale-up network and one interface for connecting to a scale-out network;
and
a scale-out network built using radix-64 switches that use dual-port Small Form Factor Pluggable (SFP) connectors or a scale-out network built using radix-128 switches that use dual-port SFP connectors,
wherein, when the scale-out network uses radix-64 switches, the scale-out network comprises:
the set of processors grouped in a railpod comprising 8 groups of 72 processors;
18 leaf switches for connecting each group of 72 processors to the 18 leaf switches via four links, in which each leaf switch is a network rail;
groups of 18 spine switches for connecting to the 18 leaf switches, in which each group of 18 spine switches is a network rail corresponding to the network rail of its connected leaf switch;
and
groups of 16 superspine switches for connecting to the groups of 18 spine switches, in which each group of 16 superspine switches is configured to connect with a corresponding group of 18 spine switches;
and
wherein, when the scale-out network uses radix-128 switches, the scale-out network comprises:
the set of processors grouped in a railpod comprising 32 groups of 72 processors;
36 leaf switches for connecting each group of 72 processors to the 36 leaf switches via two links, in which each leaf switch is a network rail;
groups of 36 spine switches for connecting to the 36 leaf switches, in which each group of 36 spine switches is a network rail corresponding to the network rail of its connected leaf switch;
and
groups of 32 superspine switches for connecting to the groups of 36 spine switches, in which each group of 32 superspine switches is configured to connect with a corresponding group of 36 spine switches.

13 more elements

Dependencies are inferred from the claim text.

Topologies for scale-out fabrics

Life of this patent

Topologies for scale-out fabrics

Life of this patent

Abstract

Claims20