# US 12,216,603 B2 · Reconfigurable peripheral component interconnect express (PCIe) data path transport to remote computing assets > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,216,603 B2 - Document ID: US12216603B2 - Status: Granted - Kind: B2 - Application number: 18/408,391 - Filed: 2024-01-09 - Published: 2024-05-02 - Granted: 2025-02-04 - Earliest claimed: 2021-11-11 - Assignees: Drut Technologies Inc. - Applicants: Drut Technologies Inc. - Inventors: Jitender Miglani; Will Ferry; Dileep Desai - Web page: https://patentgrep.com/patent/US12216603B2 ## CPC - G06F 13/4027: {using bus bridges (G06F13/4022 takes precedence)} - G02B 6/3518: {the reflective optical element being an intrinsic part of a MEMS device, i.e. fabricated together with the MEMS device (MEMS devices in general B81B; manufacture of MEM devices in general B81C; micromechanical devices controlling the direction of light G02B26/0833)} - G02B 6/3546: {NxM switch, i.e. a regular array of switches elements of matrix type constellation} - G06F 13/4022: {using switching circuits, e.g. switching matrix, connection or expansion network (G06F13/4009 takes precedence)} - G06F 13/4282: {on a serial bus, e.g. I2C bus, SPI bus (on daisy chain buses G06F13/4247)} - G06F 2213/0026: PCI express - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS ## Abstract Described are methods for configuring computing system for and computing systems for PCIe communication between remote computing assets. The system uses a fabric interface device configured to receive multi-lane serial PCIe data from functional elements of a computing asset through a multi-lane PCIe bus, and to transparently extend the multi-lane PCIe bus by converting the multi-lane PCIe data into a retimed parallel version of the PCIe multi-lane data to be sent on bidirectional data communication paths. The fabric interface device is also configured so that the multi-lane PCIe bus can have a first number of lanes and the bidirectional data communication paths can have a different second number of lanes. ## Claims ### Claim 1 A computing system comprising: a first fabric interface device coupled to receive first multi-lane serial peripheral component interconnect express (PCIe) data from first functional elements of a first computing asset through a first multi-lane PCIe bus, wherein the first fabric interface device is configured to transparently extend the first multi-lane PCIe bus by converting the first multi-lane PCIe data into a retimed parallel version of the first multi-lane PCIe data to be sent on a first bidirectional data communication path, wherein the first multi-lane PCIe bus has a first number of lanes and the first bidirectional data communication path has a second number of lanes; wherein the first number does not equal the second number, wherein the retimed parallels version of the first multi-lane PCIe data has a higher clock domain or frequency than the first multi-lane PCIe data; and wherein the first number of lanes is less than the second number of lanes; wherein the first bidirectional data communication path is configured to be connected through a data path transport to a second bidirectional data communication path; and a second fabric interface device coupled to receive multi-lane serial peripheral component interconnect express (PCIe) data from second functional elements of a second computing asset through a second multi-lane PCIe bus, wherein the second fabric interface device is configured to transparently extend the second multi-lane PCIe bus by converting the second multi-lane PCIe data into a second retimed parallel version of the second multi-lane PCIe data to be sent on the second bidirectional data communication path, wherein the second multi-lane PCIe bus has a third number of lanes and the second bidirectional data communication path has a fourth number of lanes; wherein the third number does not equal the fourth number. ### Claim 2 (depends on 1) The system of claim 1, wherein the first fabric interface device is configured to transparently extend the first multi-lane PCIe bus between the first computing asset and two other computing assets of at least three computing assets via a plurality of bidirectional data communications paths and the data path transport by converting the first multi-lane PCIe data into the retimed parallel version of the first PCIe multi-lane data to be sent on the first bidirectional data communication path through the data path transport and to the two other computing assets. ### Claim 3 (depends on 2) The system of claim 2, wherein the first computing asset is connected through the data path transport and the plurality of bidirectional data communication paths to form a bidirectional network having a bidirectional data communication path between each pair of the first and the two other computing assets of the at least three computing assets; and wherein the first fabric interface device is configured to transparently extend the first multi-lane PCIe bus by converting the retimed parallel version of the first PCIe multi-lane data received from the first bidirectional data communication path into the first multi-lane PCIe data to be sent to the first functional elements. ### Claim 4 (depends on 3) The system of claim 3, further comprising a configurable computing cluster comprising: the data path transport coupled to the first fabric interface device by the first bidirectional data communication path; the second computing asset having second functional elements and coupled to the data path transport by a second bidirectional data communication path; and the third computing asset having third functional elements and coupled to the data path transport by a third bidirectional data communication path. ### Claim 5 A computing system comprising: a fabric interface device coupled to receive multi-lane serial peripheral component interconnect express (PCIe) data from first functional elements of a first computing asset through a multi-lane PCIe bus, wherein the fabric interface device is configured to transparently extend the multi-lane PCIe bus by converting the multi-lane PCIe data into a retimed parallel version of the PCIe multi-lane data to be sent on a first bidirectional data communication path, wherein the multi-lane PCIe bus has a first number of lanes and the first bidirectional data communication path has a second number of lanes; wherein the first number does not equal the second number, wherein the retimed parallels version of the PCIe multi-lane data has a higher clock domain or frequency than the multi-lane PCIe data; and wherein the first number of lanes is less than the second number of lanes; wherein the first bidirectional data communication path is configured to be connected through a data path transport to at least three other computing assets; and wherein each of the at least three computing assets is located in a different computing device from each other of the at least three computing assets, wherein each functional element performs a function of a central processing unit (CPU), and wherein each functional element includes one of a processor chip, a chip set, a PCIe root controller, or a memory. ### Claim 6 A computing system comprising: a fabric interface device coupled to receive multi-lane serial peripheral component interconnect express (PCIe) data from first functional elements of a first computing asset through a multi-lane PCIe bus, wherein the fabric interface device is configured to transparently extend the multi-lane PCIe bus by converting the multi-lane PCIe data into a retimed parallel version of the PCIe multi-lane data to be sent on a first bidirectional data communication path, wherein the multi-lane PCIe bus has a first number of lanes and the first bidirectional data communication path has a second number of lanes; wherein the first number does not equal the second number, wherein the retimed parallels version of the PCIe multi-lane data has a higher clock domain or frequency than the multi-lane PCIe data; and wherein the first number of lanes is less than the second number of lanes; wherein the first bidirectional data communication path is configured to be connected through a data path transport to at least three other computing assets; and wherein transparently extending the multi-lane PCIe bus comprises transparently extending the multi-lane PCIe bus between the at least three computing assets via the data communications paths and the data path transport without the first, second or third functional elements being aware of the fabric interface device or of the data communication paths and without changing the PCIe data or PCIe control signals in the PCIe bus. ### Claim 7 A computing system comprising: a fabric interface device coupled to receive multi-lane serial peripheral component interconnect express (PCIe) data from first functional elements of a first computing asset through a multi-lane PCIe bus, wherein the fabric interface device is configured to transparently extend the multi-lane PCIe bus by converting the multi-lane PCIe data into a retimed parallel version of the PCIe multi-lane data to be sent on a first bidirectional data communication path, wherein the multi-lane PCIe bus has a first number of lanes and the first bidirectional data communication path has a second number of lanes; wherein the first number does not equal the second number, wherein the retimed parallels version of the PCIe multi-lane data has a higher clock domain or frequency than the multi-lane PCIe data; and wherein the first number of lanes is less than the second number of lanes; and a configurable computing cluster comprising the first bidirectional data communication path connected through a data path transport to at least three other computing assets; wherein the configurable computing cluster further comprises: a fabric manager computing device coupled to the data path transport and the at least three computing assets, wherein the fabric manager computing device is configured to: receive a request for a compute node; select, from the at least three computing assets, at least two computing assets to implement the compute node; define communications paths between the at least two computing assets in the compute node; configure the at least two computing assets; and configure the data path transport to communicate the retimed parallel version of the PCIe multi-lane data. ### Claim 8 A computing system comprising: a fabric interface device coupled to receive multi-lane serial peripheral component interconnect express (PCIe) data from first functional elements of a first computing asset through a multi-lane PCIe bus, wherein the fabric interface device is configured to transparently extend the multi-lane PCIe bus by converting the multi-lane PCIe data into a retimed parallel version of the PCIe multi-lane data to be sent on a first bidirectional data communication path, wherein the multi-lane PCIe bus has a first number of lanes and the first bidirectional data communication path has a second number of lanes; wherein the first number does not equal the second number, wherein the retimed parallels version of the PCIe multi-lane data has a higher clock domain or frequency than the multi-lane PCIe data; and wherein the first number of lanes is less than the second number of lanes; wherein the first bidirectional data communication path is configured to be connected through a data path transport to at least three computing assets; and a fabric manager computing device coupled to the data path transport and the at least three computing assets, wherein the fabric manager computing device is configured to: receive a request to execute an application; define a compute node required to execute the application; select, from the at least three computing assets, a at least two computing assets to implement the compute node; define communications paths between the at least two computing assets in the compute node; configure the second plurality of computing assets; and configure the data path transport to communicate the retimed parallel version of the PCIe multi-lane data. ### Claim 9 A reconfigurable computing cluster with remote computing assets, comprising: at least three computing assets, wherein each of the computing assets is configured to be connected to each of the other computing assets through a computer network interface card, wherein each of the at least three computing assets comprises: a functional element that performs a function of a central processing unit (CPU); and a fabric interface device coupled to the functional element via a multi-lane peripheral component interconnect express (PCIe) bus, wherein the fabric interface device is configured to transparently extend the multi-lane PCIe bus between the computing assets via fiber optic communications paths by converting PCIe data received from the PCIe bus into a retimed parallel optical version of the PCIe data, and further wherein each of the at least three computing assets is configured to be connected by the three or more bidirectional fiber optic communication paths to form a bidirectional network having a fiber optic communication path between each pair of the at least three computing assets; and a fabric manager computing device coupled to the optical circuit switch and the at least three computing assets, wherein the fabric manager computing device is configured to: receive a request for a compute node; select, from the at least three computing assets, at least two computing assets to implement the compute node; define communications paths between the at least two computing assets in the compute node; configure the at least two computing assets; and configure the optical circuit switch to provide the defined communications paths. ### Claim 10 (depends on 9) The reconfigurable computing cluster of claim 9, wherein the fabric interface device is a fabric interface card (FIC) having a field programmable gate array (FPGA) that is programmed to retime PCIe data received from the PCIe bus into the retimed parallel optical version of the PCIe data using an electronic first in first out (FIFO) circuit and an electrical to optical serializer/de-serializer (SERDES) circuit. ### Claim 11 (depends on 9) The reconfigurable computing cluster of claim 9, further comprising a photonic cross-connect switch having a photonic cross connection directly optically connecting each fabric interface device to each of the other fabric interface devices using a plurality of optical lanes upon which the retimed parallel optical version of the PCIe data is transmitted. ### Claim 12 (depends on 9) The reconfigurable computing cluster of claim 9, wherein each of the at least three computing assets is located in a different computing device from each other, and wherein being located in a different computing device is at least one of being located in a different computing device chassis, located in a different computing equipment rack or located in a different room. ### Claim 13 (depends on 9) The reconfigurable computing cluster of claim 9, wherein each fabric interface device receives the PCIe data in a first number of PCIe lanes and outputs the retimed parallel optical version of the PCIe data in a second number of optical lanes; wherein the first number does not equal the second number. ### Claim 14 (depends on 9) The reconfigurable computing cluster of claim 9, wherein the at least three computing assets comprises some or all of central processor unit assets, graphics processor unit assets, application accelerator assets, solid state drive assets, memory assets, storage assets, and network interface assets. ### Claim 15 (depends on 9) The reconfigurable computing cluster of claim 9, wherein the multi-lane PCIe bus has a first number of lanes and the bidirectional path has a second number of lanes; wherein the first number does not equal the second number; wherein the retimed parallels version of the PCIe multi-lane data has a higher clock domain or frequency than the multi-lane PCIe data; and wherein the first number of lanes is less than the second number of lanes. ### Claim 16 A method for managing a reconfigurable computing cluster comprising: providing at least four computing assets, wherein each of the at least four computing assets comprises a functional element, and a fabric interface device coupled to the functional element via a multi-lane peripheral component interconnect express (PCIe) bus, wherein the fabric interface devices within the at least four computing assets are configured to transparently extend the multi-lane PCIe bus by converting the multi-lane PCIe data into a retimed parallel version of the PCIe multi-lane data to be sent on data communications paths, and further wherein each of the at least four computing assets is connected to a data path transport to form a bidirectional network having a data communication path between each pair of the at least four computing assets; receiving a request for a compute node; selecting, from the at least four computing assets, three computing assets to implement the compute node; defining a defined data communications path between each pair of the three computing assets to implement the compute node; the data communication paths connecting each of the three computing assets to each of the other of the three computing assets; and configuring the data path transport to provide the defined data communications paths between each pair of the three computing assets. ### Claim 17 (depends on 16) The method of claim 16, wherein converting the multi-lane PCIe data into a retimed parallel version of the PCIe multi-lane data includes changing a first number of lanes of the multi-lane PCIe bus PCIe electronic data to a different number of lanes of the retimed parallel version of the PCIe multi-lane data for transmission on the bidirectional data communication paths; wherein the retimed parallels version of the PCIe multi-lane data has a higher clock domain or frequency than the multi-lane PCIe data; and wherein the first number of lanes is less than the different number of lanes. ### Claim 18 (depends on 17) The method of claim 17, wherein receiving a request for a compute node comprises: receiving a request to execute an application; and defining a compute node required to execute the application. ### Claim 19 (depends on 18) The method of claim 18, further comprising changing the different number of lanes of the retimed parallel version of the PCIe multi-lane data into optical data; wherein: each asset is one of: a CPU asset, a network interface asset, a GPU asset, a memory asset, or a solid state drive asset; each data communication path is a bidirectional optical fiber; the data path transport protocol is a photonic cross-connect switch; and transparently extending the multi-lane PCIe bus comprises transparently extending the multi-lane PCIe bus between each computing assets via the data communications paths and the data path transport without any of the functional elements being aware of the fabric interface device or of the data communication paths and without changing the PCIe data or changing PCIe control signals in the PCIe bus.