# US 11,907,151 B2 · Reconfigurable peripheral component interconnect express (PCIe) data path transport to remote computing assets > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,907,151 B2 - Document ID: US11907151B2 - Status: Granted - Kind: B2 - Application number: 17/524,641 - Filed: 2021-11-11 - Published: 2023-05-11 - Granted: 2024-02-20 - Earliest claimed: 2021-11-11 - Assignees: Drut Technologies Inc. - Applicants: Drut Technologies Inc. - Inventors: Jitender Miglani; Will Ferry; Dileep Desai - Web page: https://patentgrep.com/patent/US11907151B2 ## CPC - G06F 13/4027: {using bus bridges (G06F13/4022 takes precedence)} - G02B 6/3518: {the reflective optical element being an intrinsic part of a MEMS device, i.e. fabricated together with the MEMS device (MEMS devices in general B81B; manufacture of MEM devices in general B81C; micromechanical devices controlling the direction of light G02B26/0833)} - G02B 6/3546: {NxM switch, i.e. a regular array of switches elements of matrix type constellation} - G06F 13/4022: {using switching circuits, e.g. switching matrix, connection or expansion network (G06F13/4009 takes precedence)} - G06F 13/4282: {on a serial bus, e.g. I2C bus, SPI bus (on daisy chain buses G06F13/4247)} - G06F 2213/0026: PCI express - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS ## Abstract Described are methods for configuring computing system for and computing systems for PCIe communication between remote computing assets. The system uses a fabric interface device configured to receive multi-lane serial PCIe data from functional elements of a computing asset through a multi-lane PCIe bus, and to transparently extend the multi-lane PCIe bus by converting the multi-lane PCIe data into a retimed parallel version of the PCIe multi-lane data to be sent on bidirectional data communication paths. The fabric interface device is also configured so that the multi-lane PCIe bus can have a first number of lanes and the bidirectional data communication paths can have a different second number of lanes. ## Claims ### Claim 1 A computing system comprising: a first fabric interface device coupled to receive first multi-lane serial peripheral component interconnect express (PCIe) data from first functional elements of a first computing asset through a first multi-lane PCIe bus, wherein the first fabric interface device is configured to transparently extend the first multi-lane PCIe bus by converting the first multi-lane PCIe data into a retimed parallel version of the first PCIe multi-lane data to be sent on a first bidirectional data communication path, wherein the first multi-lane PCIe bus has a first number of lanes and the first bidirectional data communication path has a second number of lanes; and wherein the first number does not equal the second number; the first computing asset configured to be coupled to a data path transport by the first bidirectional data communication path; and a second computing asset having second functional elements and configured to be coupled to the data path transport by a second bidirectional data communication path, wherein the second computing asset includes: a second fabric interface device coupled to receive second multi-lane serial peripheral component interconnect express (PCIe) data from the second functional elements through a second multi-lane PCIe bus, wherein the second fabric interface device is configured to transparently extend the second multi-lane PCIe bus by converting the second multi-lane PCIe data into a second retimed parallel version of the second PCIe multi-lane data to be sent on the second bidirectional data communication path, wherein the second multi-lane PCIe bus has a third number of lanes and the second bidirectional data communication path has a fourth number of lanes; and wherein the third number does not equal the fourth number. ### Claim 2 (depends on 1) The system of claim 1, wherein the first fabric interface device is configured to transparently extend the first multi-lane PCIe bus between the first computing asset and two other computing assets of a first at least three computing assets via a plurality of bidirectional data communications paths and the data path transport by converting the first multi-lane PCIe data into the retimed parallel version of the first PCIe multi-lane data to be sent on the first bidirectional data communication path through the data path transport and to the two other computing assets. ### Claim 3 (depends on 2) The system of claim 2, wherein the first computing asset is connected through the data path transport and the plurality of bidirectional data communication paths to form a bidirectional network having a bidirectional data communication path between each pair of the first and the two other computing assets of the at least three computing assets; and wherein the first fabric interface device is configured to transparently extend the first multi-lane PCIe bus by converting the retimed parallel version of the first PCIe multi-lane data received from the first bidirectional data communication path into the first multi-lane PCIe data to be sent to the first functional elements. ### Claim 4 (depends on 3) The system of claim 3, further comprising a configurable computing cluster comprising: the data path transport coupled to the first fabric interface device by the first bidirectional data communication path; the second computing asset coupled to the data path transport by the second bidirectional data communication path; and the third computing asset having third functional elements and coupled to the data path transport by a third bidirectional data communication path. ### Claim 5 (depends on 1) The system of claim 1, wherein the retimed parallels version of the PCIe multi-lane data has a higher clock domain or frequency than the multi-lane PCIe data; and wherein the first number of lanes is less than the second number of lanes. ### Claim 6 A computing system comprising: a first fabric interface device coupled to receive first multi-lane serial peripheral component interconnect express (PCIe) data from first functional elements of a first computing asset through a first multi-lane PCIe bus, wherein the first fabric interface device is configured to transparently extend the first multi-lane PCIe bus by converting the first multi-lane PCIe data into a retimed parallel version of the first PCIe multi-lane data to be sent on a first bidirectional data communication path, wherein the first multi-lane PCIe bus has a first number of lanes and the first bidirectional data communication path has a second number of lanes; wherein the first number does not equal the second number; wherein the first fabric interface device is configured to transparently extend the first multi-lane PCIe bus between the first computing asset and two other computing assets of a first at least three computing assets via a plurality of bidirectional data communications paths and the data path transport by converting the first multi-lane PCIe data into the retimed parallel version of the first PCIe multi-lane data to be sent on the first bidirectional data communication path through the data path transport and to the two other computing assets; wherein each of the first at least three computing assets is located in a different computing device from each other of the first at least three computing assets, wherein each functional element performs a function of a central processing unit (CPU), and wherein each functional element includes one of a processor chip, a chip set, a PCIe root controller, or a memory. ### Claim 7 A computing system comprising: a first fabric interface device coupled to receive first multi-lane serial peripheral component interconnect express (PCIe) data from first functional elements of a first computing asset through a first multi-lane PCIe bus, wherein the first fabric interface device is configured to transparently extend the first multi-lane PCIe bus by converting the first multi-lane PCIe data into a retimed parallel version of the first PCIe multi-lane data to be sent on a first bidirectional data communication path, wherein the first multi-lane PCIe bus has a first number of lanes and the first bidirectional data communication path has a second number of lanes; wherein the first number does not equal the second number; wherein the first fabric interface device is configured to transparently extend the first multi-lane PCIe bus between the first computing asset and two other computing assets of a first at least three computing assets via a plurality of bidirectional data communications paths and the data path transport by converting the first multi-lane PCIe data into the retimed parallel version of the first PCIe multi-lane data to be sent on the first bidirectional data communication path through the data path transport and to the two other computing assets; and wherein transparently extending the first multi-lane PCIe bus comprises transparently extending the first multi-lane PCIe bus between the first at least three computing assets via the data communications paths and the data path transport without functional elements of the first at least three computing assets being aware of the first fabric interface device or of the data communication paths and without changing the PCIe data or PCIe control signals in the PCIe bus. ### Claim 8 A computing system comprising: a first fabric interface device coupled to receive first multi-lane serial peripheral component interconnect express (PCIe) data from first functional elements of a first computing asset through a first multi-lane PCIe bus, wherein the first fabric interface device is configured to transparently extend the first multi-lane PCIe bus by converting the first multi-lane PCIe data into a retimed parallel version of the first PCIe multi-lane data to be sent on a first bidirectional data communication path, wherein the first multi-lane PCIe bus has a first number of lanes and the first bidirectional data communication path has a second number of lanes; wherein the first number does not equal the second number; and wherein the first fabric interface device is configured to transparently extend the first multi-lane PCIe bus between the first computing asset and two other computing assets of a first at least three computing assets via a plurality of bidirectional data communications paths and the data path transport by converting the first multi-lane PCIe data into the retimed parallel version of the first PCIe multi-lane data to be sent on the first bidirectional data communication path through the data path transport and to the two other computing assets; and a configurable computing cluster comprising: a fabric manager computing device coupled to the data path transport and the first at least three computing assets, wherein the fabric manager computing device is configured to: receive a request for a compute node; select, from the first at least three computing assets, a second at least three computing assets to implement the compute node; define communications paths between the second at least three computing assets in the compute node; configure the second at least three computing assets; and configure the data path transport to communicate the retimed parallel version of the PCIe multi-lane data. ### Claim 9 A computing system comprising: a first fabric interface device coupled to receive first multi-lane serial peripheral component interconnect express (PCIe) data from first functional elements of a first computing asset through a first multi-lane PCIe bus, wherein the first fabric interface device is configured to transparently extend the first multi-lane PCIe bus by converting the first multi-lane PCIe data into a retimed parallel version of the first PCIe multi-lane data to be sent on a first bidirectional data communication path, wherein the first multi-lane PCIe bus has a first number of lanes and the first bidirectional data communication path has a second number of lanes; wherein the first number does not equal the second number; and wherein the first fabric interface device is configured to transparently extend the first multi-lane PCIe bus between the first computing asset and two other computing assets of a first at least three computing assets via a plurality of bidirectional data communications paths and the data path transport by converting the first multi-lane PCIe data into the retimed parallel version of the first PCIe multi-lane data to be sent on the first bidirectional data communication path through the data path transport and to the two other computing assets; and a fabric manager computing device coupled to the data path transport and the first at least three computing assets, wherein the fabric manager computing device is configured to: receive a request to execute an application; define a compute node required to execute the application; select, from the first at least three computing assets, a second at least three computing assets to implement the compute node; define communications paths between the second at least three computing assets in the compute node; configure the second plurality of computing assets; and configure the data path transport to communicate the retimed parallel version of the first PCIe multi-lane data. ### Claim 10 A reconfigurable computing cluster with remote computing assets, comprising: an photonic cross-connect switch; three or more bidirectional fiber optic communication paths; and a first of at least three computing assets, wherein each of the computing assets is connected to each of the other computing assets through a computer network interface card, wherein each of the first at least three computing assets comprises: functional elements, wherein each functional element performs a function of a central processing unit (CPU); and a fabric interface device coupled to the functional elements via a multi-lane peripheral component interconnect express (PCIe) bus, wherein the fabric interface device within the first at least three computing assets are configured to transparently extend the multi-lane PCIe bus between computing assets via the fiber optic communications paths and the optical circuit switch by converting PCIe data received from the PCIe bus into a retimed parallel optical version of the PCIe data, and further wherein each of the first at least three computing assets is connected to the optical circuit switch by the three or more bidirectional fiber optic communication paths to form a bidirectional network having a communication path between each pair of the first at least three computing assets. ### Claim 11 (depends on 10) The reconfigurable computing cluster of claim 10, wherein the fabric interface device is a fabric interface card (FIC) having a field programmable gate array (FPGA) that is programmed to retime PCIe data received from the PCIe bus into the retimed parallel optical version of the PCIe data using an electronic first in first out (FIFO) circuit and an electrical to optical serializer/de-serializer (SERDES) circuit. ### Claim 12 (depends on 10) The reconfigurable computing cluster of claim 10, wherein the photonic cross-connect switch has a photonic cross connection directly optically connecting each fabric interface device to each of the other fabric interface devices using a plurality of optical lanes upon which the retimed parallel optical version of the PCIe data is transmitted. ### Claim 13 (depends on 10) The reconfigurable computing cluster of claim 10, wherein being located in a different computing device is at least one of being located in a different computing device chassis, located in a different computing equipment rack or located in a different room. ### Claim 14 (depends on 10) The reconfigurable computing cluster of claim 10, wherein each fabric interface device receives the PCIe data in a first number of PCIe lanes and outputs the retimed parallel optical version of the PCIe data in a second number of optical lanes; wherein the first number does not equal the second number. ### Claim 15 (depends on 10) The reconfigurable computing cluster of claim 10, wherein the first at least three computing assets comprises some or all of central processor unit assets, graphics processor unit assets, application accelerator assets, solid state drive assets, memory assets, storage assets, and network interface assets. ### Claim 16 (depends on 10) The reconfigurable computing cluster of claim 10, further comprising: a fabric manager computing device coupled to the optical circuit switch and the first at least three computing assets, wherein the fabric manager computing device is configured to: receive a request for a compute node; select, from the first at least three computing assets, a second at least three computing assets to implement the compute node; define communications paths between the second at least three computing assets in the compute node; configure the second at least three computing assets; and configure the optical circuit switch to provide the defined communications paths. ### Claim 17 A method for managing a reconfigurable computing cluster comprising: providing a reconfigurable computing cluster, comprising a data path transport, at least four computing assets, and two or more bidirectional data communication paths coupled to each computing asset, wherein each of the first plurality of computing assets comprises functional elements, and a fabric interface device coupled to the functional elements via a multi-lane peripheral component interconnect express (PCIe) bus, wherein the fabric interface devices within the at least four computing assets are configured to transparently extend the multi-lane PCIe bus by converting the multi-lane PCIe data into a retimed parallel version of the PCIe multi-lane data to be sent on the data communications paths, and further wherein each of the at least four computing assets is connected to the data path transport to form a bidirectional network having a data communication path between each pair of the at least four computing assets; receiving a request for a compute node; selecting, from the at least four computing assets, three computing assets to implement the compute node; defining a defined data communications path between each pair of the three computing assets to implement the compute node; the data communication paths connecting each of the three computing assets to each of the other of the three computing assets; and configuring the data path transport to provide the defined data communications paths between each pair of the three computing assets. ### Claim 18 (depends on 17) The method of claim 17, wherein converting the multi-lane PCIe data into a retimed parallel version of the PCIe multi-lane data includes changing a first number of lanes of the multi-lane PCIe bus PCIe electronic data to a different number of lanes of the retimed parallel version of the PCIe multi-lane data for transmission on the bidirectional data communication paths. ### Claim 19 (depends on 18) The method of claim 18, wherein receiving a request for a compute node comprises: receiving a request to execute an application; and defining a compute node required to execute the application. ### Claim 20 (depends on 19) The method of claim 19, further comprising changing the different number of lanes of the retimed parallel version of the PCIe multi-lane data into optical data; wherein: each asset is one of: a CPU asset, a network interface asset, a GPU asset, a memory asset, or a solid state drive asset; each data communication path is a bidirectional optical fiber; the data path transport protocol is a photonic cross-connect switch; and transparently extending the multi-lane PCIe bus comprises transparently extending the multi-lane PCIe bus between each computing assets via the data communications paths and the data path transport without any of the functional elements being aware of the fabric interface device or of the data communication paths and without changing the PCIe data or changing PCIe control signals in the PCIe bus.