# US 2018/0322082 A1 · Peer-To-Peer Communication For Graphics Processing Units

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0322082 A1
- Document ID: US20180322082A1
- Status: Published application
- Kind: A1
- Application number: 15/848,268
- Filed: 2017-12-20
- Published: 2018-11-08
- Earliest claimed: 2017-05-08
- Assignees: Liqid Inc.
- Applicants: Liqid Inc.
- Inventors: Jason Breakstone; German Kazakov; Christopher R. Long; James Scott Cannata
- Web page: https://patentgrep.com/patent/US20180322082A1

## CPC

- G06F 13/4022: {using switching circuits, e.g. switching matrix, connection or expansion network (G06F13/4009 takes precedence)}
- G06F 9/5044: {considering hardware capabilities}
- G06F 9/5077: {Logical partitioning of resources; Management or configuration of virtualized resources (specific details on emulation or internal functioning of virtual machines G06F9/455)}
- G06F 12/02: Addressing or allocation; Relocation (program address sequencing G06F9/00; arrangements for selecting an address in a digital store G11C8/00)
- G06F 13/28: using burst mode transfer, e.g. direct memory access {DMA}, cycle steal (G06F13/32 takes precedence)
- G06F 13/4282: {on a serial bus, e.g. I2C bus, SPI bus (on daisy chain buses G06F13/4247)}
- G06T 1/20: Processor architectures; Processor configuration, e.g. pipelining
- G06F 2213/0026: PCI express
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Disaggregated computing architectures, platforms, and systems are provided herein. In one example, a method of operating a data processing system is provided. The method includes communicatively coupling graphics processing units (GPUs) over a Peripheral Component Interconnect Express (PCIe) fabric. The method also includes establishing a peer-to-peer arrangement between the GPUs over the PCIe fabric by at least providing an isolation function in the PCIe fabric configured to isolate a device PCIe address domain associated with the GPUs from at least a local PCIe address domain associated with a host processor that initiates the peer-to-peer arrangement between the GPUs.

## Claims

### Claim 1

A method of operating a data processing system, the method comprising:
communicatively coupling graphics processing units (GPUs) over a Peripheral Component Interconnect Express (PCIe) fabric; and
establishing a peer-to-peer arrangement between the GPUs over the PCIe fabric by at least providing an isolation function in the PCIe fabric configured to isolate a device PCIe address domain associated with the GPUs from at least a local PCIe address domain associated with a host processor that initiates the peer-to-peer arrangement between the GPUs.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
isolating the device PCIe address domain from the local PCIe address domain by at least establishing synthetic PCIe devices representing the GPUs in the local PCIe address domain; and
redirecting traffic transferred by the host processor for the GPUs in the local PCIe address domain for delivery to corresponding ones of the GPUs in the device PCIe address domain.

### Claim 3 (depends on 2)

The method of claim 2, further comprising:
dynamically adding a selected GPU into the peer-to-peer arrangement by at least instantiating the selected GPU into the device PCIe address domain and providing a further synthetic device representing the selected GPU on the local PCIe address domain.

### Claim 4 (depends on 3)

The method of claim 3, further comprising:
dynamically removing the selected GPU from the peer-to-peer arrangement by at least inactivating further synthetic device.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
establishing the isolation function in the PCIe fabric by at least providing address traps that monitor for traffic over the PCIe fabric directed to PCIe addresses associated with the GPUs in the local PCIe address domain and translate the PCIe addresses into corresponding PCIe addresses for the GPUs in the device PCIe address domain.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
responsive to peer-to-peer traffic from a first of the GPUs indicating the second of the GPUs as a destination in the local PCIe address domain, employing a peer-to-peer address trap in the isolation function to receive the peer-to-peer traffic and transfer the peer-to-peer traffic to the second of the GPUs in the device PCIe address domain.

### Claim 7 (depends on 6)

The method of claim 6, wherein the peer-to-peer arrangement between the GPUs comprises a direct memory access (DMA) link established between memory associated with each of the GPUs.

### Claim 8 (depends on 1)

The method of claim 1, further comprising:
in a control processor, establishing the isolation function in PCIe switch circuitry that communicatively couples the GPUs and the host processor over the PCIe fabric.

### Claim 9 (depends on 1)

The method of claim 1, wherein the isolation function comprises one or more address traps and one or more address translation tables.

### Claim 10 (depends on 1)

The method of claim 1, further comprising:
selecting PCIe addresses in the isolation function for the GPUs in the device PCIe address domain that do not overlap with PCIe address usages of further GPUs communicatively coupled on the PCIe fabric.

### Claim 11

A data processing system, comprising:
a Peripheral Component Interconnect Express (PCIe) fabric configured to communicatively couple graphics processing units (GPUs) with at least a host processor; and
a control processor configured to facilitate a peer-to-peer arrangement between the GPUs over the PCIe fabric by at least establishing an isolation function in the PCIe fabric configured to isolate a device PCIe address domain associated with the GPUs from at least a local PCIe address domain associated with a host processor that initiates the peer-to-peer arrangement between the GPUs.

### Claim 12 (depends on 11)

The data processing system of claim 11, comprising:
the control processor configured to isolate the device PCIe address domain from the local PCIe address domain by at least establishing synthetic PCIe devices representing the GPUs in the local PCIe address domain; and
the isolation function configured to redirect traffic transferred by the host processor for the GPUs in the local PCIe address domain for delivery to corresponding ones of the GPUs in the device PCIe address domain.

### Claim 13 (depends on 12)

The data processing system of claim 12, comprising:
the control processor configured to dynamically add a selected GPU into the peer-to-peer arrangement by at least instantiating the selected GPU into the device PCIe address domain and providing a further synthetic device representing the selected GPU on the local PCIe address domain; and
the control processor configured to dynamically remove the selected GPU from the peer-to-peer arrangement by at least inactivating further synthetic device.

### Claim 14 (depends on 11)

The data processing system of claim 11, comprising:
the control processor configured to establish the isolation function in the PCIe fabric by at least providing address traps that monitor for traffic over the PCIe fabric directed to PCIe addresses associated with the GPUs in the local PCIe address domain and translate the PCIe addresses into corresponding PCIe addresses for the GPUs in the device PCIe address domain.

### Claim 15 (depends on 11)

The data processing system of claim 11, comprising:
responsive to peer-to-peer traffic from a first of the GPUs indicating the second of the GPUs as a destination in the local PCIe address domain, the isolation function configured to employ a peer-to-peer address trap to receive the peer-to-peer traffic and transfer the peer-to-peer traffic to the second of the GPUs in the device PCIe address domain.

### Claim 16 (depends on 15)

The data processing system of claim 15, wherein the peer-to-peer arrangement between the GPUs comprises a direct memory access (DMA) link established between memory associated with each of the GPUs.

### Claim 17 (depends on 11)

The data processing system of claim 11, comprising:
the control processor configured to establish the isolation function in PCIe switch circuitry that communicatively couples the GPUs and the host processor over the PCIe fabric, wherein the isolation function comprises one or more address traps and one or more address translation tables.

### Claim 18 (depends on 11)

The data processing system of claim 11, comprising:
the control processor configured to select PCIe addresses in the isolation function for the GPUs in the device PCIe address domain that do not overlap with PCIe address usages of further GPUs communicatively coupled on the PCIe fabric.

### Claim 19

A data processing apparatus comprising:
one or more computer readable storage media;
a processing system operatively coupled with the one or more computer readable storage media; and
program instructions stored on the one or more computer readable storage media, that when executed by the processing system, direct the processing system to at least:
establish a peer-to-peer arrangement between graphics processing units (GPUs) over a Peripheral Component Interconnect Express (PCIe) fabric by at least providing an isolation function in the PCIe fabric configured to isolate a device PCIe address domain associated with the GPUs from at least a local PCIe address domain associated with a host processor that initiates the peer-to-peer arrangement between the GPUs;
wherein the isolation function comprises synthetic PCIe devices representing the GPUs in the local PCIe address domain;
wherein the isolation function is configured to redirect traffic transferred by the host processor for the GPUs in the local PCIe address domain for delivery to corresponding ones of the GPUs in the device PCIe address domain; and
wherein the isolation function is further configured to redirect peer-to-peer traffic transferred by a first of the GPUs indicating the second of the GPUs as a destination in the local PCIe address domain to the second of the GPUs in the device PCIe address domain.

### Claim 20 (depends on 19)

The apparatus of claim 19, comprising further program instructions, that when executed by the processing system, direct the processing system to at least:
select PCIe addresses in the isolation function for the GPUs in the device PCIe address domain that do not overlap with PCIe address usages of further GPUs communicatively coupled on the PCIe fabric.
