Reconfigurable computing pods using optical networks with one-to-many optical switches

A system, comprising:

• an optical network comprising a plurality of optical circuit switching (OCS) switches and a plurality of one-to-many optical switches;
• and
• a set of building blocks that each include an m-dimensional arrangement of compute nodes, wherein:
• each building block is connected to the optical network;
• for each dimension of the m dimensions:
• each building block includes one or more segments of interconnected compute nodes along the dimension, each segment including a first compute node on a first end of the segment and a second compute node on a second end of the segment opposite the first end;
• a first portion of the first compute nodes are connected to a first OCS switch of two or more OCS switches for the dimension;
• one or more additional portions of the first compute nodes are connected to a respective additional OCS switch of the two or more OCS switches for the dimension;
• and
• the second compute node of each segment is connected to an input of a respective one-to-many optical switch having the input and multiple outputs, wherein a first output is connected to the first OCS switch and, for each additional portion of first compute nodes, a respective additional output is connected to the additional OCS switch for the additional portion of first compute nodes.

A non-transitory computer storage medium encoded with a computer program, the program comprising instructions that when executed by one or more data processing apparatus cause the one or more data processing apparatus to perform operations comprising:

• receiving request data specifying requested compute nodes for a computing workload, the request data specifying a target arrangement of the compute nodes;
• selecting, from a superpod that includes a set of building blocks that each include an m-dimensional arrangement of compute nodes, a subset of the building blocks, wherein:
• each building block is connected to an optical network that includes two or more optical circuit switching (OCS) switches for each of the m dimensions;
• for each dimension of the m dimensions:
• each building block includes segments of interconnected compute nodes along the dimension, each segment including a first compute node on a first end of the segment and a second compute node on a second end of the segment opposite the first end;
• a first portion of the first compute nodes is connected to a first OCS switch of the two or more OCS switches for the dimension;
• one or more additional portions of the first compute nodes are connected to a respective additional OCS switch of the two or more OCS switches for the dimension;
• and
• the second compute node of each segment is connected to an input of a respective one-to-many optical switch having the input and multiple outputs, wherein a first output is connected to the first OCS switch and, for each additional portion of first compute nodes, a respective additional output is connected to the additional OCS switch for the additional portion of first compute nodes;
• determining a logical arrangement of the subset of the building blocks that matches the target arrangement of the compute nodes, the logical arrangement defining, for each of the m dimensions, connections between the segments of each building block to corresponding segments of one or more other building blocks;
• generating a workload cluster of compute nodes that includes the subset of the building blocks, wherein the compute nodes of the workload cluster are connected to each other based on the logical arrangement, and wherein the generating comprises:
• configuring, for each dimension of the workload cluster, respective routing data for each of the two or more optical circuit switches for the dimension, the respective routing data for each dimension of the workload cluster specifying how data of the computing workload is routed between compute nodes along the dimension of the workload cluster;
• and
• configuring, based on the logical arrangement, at least a portion of the one-to-many switches such that the second compute node of each segment is connected to a same OCS switch as a corresponding first compute node of a corresponding segment to which the second compute node is connected in the logical arrangement;
• and
• causing the compute nodes of the workload cluster to execute the computing workload.