# US 2021/0067449 A1 · Techniques for reducing the overhead of providing responses in a computing network

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0067449 A1
- Document ID: US20210067449A1
- Status: Published application
- Kind: A1
- Application number: 16/553,511
- Filed: 2019-08-28
- Published: 2021-03-04
- Earliest claimed: 2019-08-28
- Assignees: NVIDIA Corporation
- Applicants: Nvidia Corporation
- Inventors: Glenn Dearth; Mark Hummel; Jonathan Owen; Mike Osborn; John Wortman; Rich Reeves
- Web page: https://patentgrep.com/patent/US20210067449A1

## CPC

- G06F 13/4009: {with data restructuring}
- G06F 13/1642: {with request queuing}
- G06F 13/4022: {using switching circuits, e.g. switching matrix, connection or expansion network (G06F13/4009 takes precedence)}
- H04L 7/0008: {Synchronisation information channels, e.g. clock distribution lines}
- H04L 45/74591: {using content-addressable memories [CAM]}
- H04L 49/25: Routing or path finding in a switch fabric
- G06F 13/1689: {Synchronisation and timing concerns (synchronisation on a memory bus G06F13/4234)}
- G06F 13/30: with priority control
- G06F 13/4234: {being a memory bus}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

An endpoint in a network may make posted or non-posted write requests to another endpoint in the network. For a non-posted write request, the target endpoint provides a response to the requesting endpoint indicating that the write request has been serviced. For a posted write request, the target endpoint does not provide such an acknowledgment. Hence, posted write requests have lower overhead, but they suffer from potential synchronization and resiliency issues. While non-posted write requests do not have those issues, they cause increased load on the network because such requests require the target endpoint to acknowledge each write request. Introduced herein is a network operation technique that uses non-posted transactions while maintaining a load overhead of the network as a manageable level. The introduced technique reduces the load overhead of the non-posted write requests by collapsing and reducing a number of the responses.

## Claims

### Claim 1

A method of operating a fabric comprising:
forwarding requests from a source endpoint to a target endpoint; and
forwarding responses of the requests from the target endpoint to the source endpoint, wherein a number of the responses is less than a number of the requests.

### Claim 2 (depends on 1)

The method of claim 1, wherein the forwarding the responses includes reducing the number of the responses by collapsing some of the responses that belong to a same stream into a single collapsed response.

### Claim 3 (depends on 2)

The method of claim 2, wherein the collapsing includes:
determining that at least one of the responses is collapsible;
comparing the at least one collapsible response to other collapsible responses; and
if a stream identifier of the at least one collapsible response matches a stream identifier of one of the other collapsible responses, incrementing a collapse count in the one of the other collapsible responses and discarding the at least one collapsible response.

### Claim 4 (depends on 3)

The method of claim 3, wherein the collapsing further includes storing the at least one collapsible response if the stream identifier of the at least one collapsible response does not match stream identifiers of the other collapsible responses.

### Claim 5 (depends on 3)

The method of claim 3, wherein the collapsing further includes continuing to forward the one of the other collapsible responses to the source endpoint when a predetermined time period expires or when the collapse count reaches a maximum count or an expected response number for a stream that the one of the other collapsible responses belongs to.

### Claim 6 (depends on 2)

The method of claim 2, wherein the collapsing is performed using a content-addressable memory of a switch connected to the fabric.

### Claim 7 (depends on 1)

The method of claim 1, wherein the forwarding the requests includes changing original stream identifiers of the requests with unique stream identifiers when the original stream identifiers do not indicate streams to which the requests belong to.

### Claim 8 (depends on 7)

The method of claim 7, wherein the changing includes changing an original stream identifier of one of the requests with a unique stream identifier from a non-collapsible tag pool when the one request is not collapsible.

### Claim 9 (depends on 7)

The method of claim 7, wherein the changing includes when one of the requests is collapsible and does not belong to tracked streams in a memory, changing an original stream identifier of the one request with a unique stream identifier from a collapsible tag pool and storing a stream, with which the unique stream identifier is associated, in the memory

### Claim 10 (depends on 7)

The method of claim 7, wherein the changing includes changing an original stream identifier of one of the requests with a unique stream identifier from a non-collapsible tag pool when the one request is collapsible and a stream, to which the one request belongs, cannot be tracked.

### Claim 11 (depends on 7)

The method of claim 7, wherein the changing includes changing an original stream identifier of one of the requests with a unique stream identifier associated with a tracked stream and incrementing a collapse count of the tracked stream when the one request is collapsible and belongs to the tracked stream.

### Claim 12 (depends on 1)

The method of claim 1 further comprising synchronizing the source endpoint and the target endpoint by tracking flush requests and responding to each of the flush requests when all transactions before the each flush request are completed.

### Claim 13 (depends on 12)

The method of claim 12, wherein the tracking includes:
incrementing an open counter of a tracking structure when a request of at least one stream arrives at the tracking structure and decrementing the open counter when a response of the at least one stream arrives at the tracking structure;
receiving one of the flush requests at the tracking structure; and
upon the receiving:
if a closed counter of the tracking structure is empty or zero, transitioning the open and closed counters to opposite states; and
if the closed counter has not reached zero, marking the open and closed counters to transition to the opposite states when the closed counter reaches zero and continuing to track incoming transaction and flush request on the open counter.

### Claim 14 (depends on 13)

The method of claim 13, wherein the tracking further includes when the closed counter reaches zero, responding to all flush requests that arrived while the closed counter was the open counter and transitioning the open and closed counters to the opposite states.

### Claim 15 (depends on 14)

The method of claim 14, wherein the transitioning the open and closed counters to the opposite states includes updating extended stream identifiers of transactions to be tracked by a new open counter.

### Claim 16 (depends on 1)

The method of claim 1, further comprising detecting an error in a particular stream by determining whether all transactions of the particular stream are completed within a predetermined time period.

### Claim 17 (depends on 16)

The method of claim 16, wherein the determining includes tracking responses of the particular stream using a closed counter of a source-track structure, and when the closed counter does not reach zero before a timer associated the closed counter expires, notifying the source endpoint that the error has occurred in the particular stream.

### Claim 18 (depends on 17)

The method of claim 17, wherein the notifying includes clearing transactions of the particular stream from the source endpoint and the fabric.

### Claim 19 (depends on 17)

The method of claim 17, wherein a usage of the timer is time-multiplexed and shared by multiple closed counters.

### Claim 20 (depends on 17)

The method of claim 17, wherein a usage of the closed counter is time-multiplexed and shared by multiple open counters.

### Claim 21 (depends on 1)

The method of claim 1, wherein the requests are non-posted write requests.

### Claim 22

A device for operating a fabric comprising:
a pipeline configured to forward requests from a source endpoint to a target endpoint and forward responses of the requests from the target endpoint to the source endpoint; and
a collapsing structure connected to the pipeline and configured to reduce a load of the responses on the fabric, wherein a number of the responses is less than a number of the requests.

### Claim 23 (depends on 22)

The device of claim 22, wherein the collapsing structure is configured to reduce the load of the responses on the fabric by collapsing some of the responses that belong to a same stream into a single collapsed response.

### Claim 24 (depends on 23)

The device of claim 23, wherein the collapsing includes:
determining that at least one of the responses is collapsible;
comparing the at least one collapsible response to other collapsible responses that are stored in the collapsing structure; and
if a stream identifier of the at least one collapsible response matches a stream identifier of one of the other collapsible responses, incrementing a collapse count in the one of the other collapsible responses and discarding the at least one collapsible response.

### Claim 25 (depends on 24)

The device of claim 24, wherein the collapsing further includes storing the at least one collapsible response in the collapsing structure if the stream identifier of the at least one collapsible response does not match stream identifiers of the other collapsible responses and the collapsing structure has an available slot.

### Claim 26 (depends on 24)

The device of claim 24, wherein the collapsing structure is further configured to release the one of the other collapsible responses into the pipeline when a predetermined time period expires or when the collapse count reaches a maximum count or an expected response number for a stream that the one of the other collapsible responses belongs to.

### Claim 27 (depends on 22)

The device of claim 22, wherein the collapsing structure is a content-addressable memory.

### Claim 28 (depends on 22)

The device of claim 22 further comprising a tag remapping structure configured to change original stream identifiers of the requests with unique stream identifiers when the original stream identifiers do not indicate streams, to which the requests belong.

### Claim 29 (depends on 28)

The device of claim 28, wherein an original stream identifier of one of the requests is changed with a unique stream identifier from a non-collapsible tag pool when the one request is not collapsible.

### Claim 30 (depends on 28)

The device of claim 28, wherein when one of the requests is collapsible and does not belong to tracked streams in the tag remapping structure, an original stream identifier of the one request is changed with a unique stream identifier from a collapsible tag pool and a stream, with which the unique stream identifier is associated, is stored in the tag remapping structure.

### Claim 31 (depends on 28)

The device of claim 28, wherein an original stream identifier of one of the requests is changed with a unique stream identifier from a non-collapsible tag pool when the one request is collapsible and a stream, to which the one request belongs, cannot be tracked by the tag remapping structure.

### Claim 32 (depends on 28)

The device of claim 28, wherein an original stream identifier of one of the requests is changed with a unique stream identifier associated with a tracked stream in the tag remapping structure when the one request is collapsible and belongs to the tracked stream.

### Claim 33 (depends on 22)

The device of claim 22 further comprising a tracking structure configured to synchronize the source endpoint and the target endpoint by tracking flush requests and responding to each of the flush requests when all transactions before the each flush request are completed.

### Claim 34 (depends on 33)

The device of claim 33, wherein the tracking structure includes an open counter and a closed counter and the tracking includes:
incrementing the open counter when a request of at least one stream arrives at the tracking structure and decrementing the open counter when a response of the at least one stream arrives at the tracking structure;
receiving one of the flush requests at the tracking structure;
upon the receiving:
if the closed counter is empty or zero, transitioning the open and closed counters to opposite states; and
if the closed counter has not reached zero, marking the open and closed counters to transition to the opposite states when the closed counter reaches zero and continuing to track incoming transaction and flush request on the open counter.

### Claim 35 (depends on 34)

The device of claim 34, wherein the tracking further includes when the closed counter reaches zero, responding to all flush requests that arrived while the closed counter was the open counter and transitioning the open and closed counters to the opposite states.

### Claim 36 (depends on 35)

The device of claim 35, wherein the transitioning the open and closed counters to the opposite states includes updating extended stream identifiers of transactions to be tracked by a new open counter.

### Claim 37 (depends on 22)

The device of claim 22, further comprising a source-track structure configured to detect an error in a particular stream by determining whether all transactions of the particular stream are completed within a predetermined time period.

### Claim 38 (depends on 37)

The device of claim 37, wherein the determining includes tracking responses of the particular stream using a closed counter of the source-track structure, and when the closed counter does not reach zero before a timer associated the closed counter expires, notifying the source endpoint that the error has occurred in the particular stream.

### Claim 39 (depends on 38)

The device of claim 38, wherein a usage of the timer is time-multiplexed and shared by multiple closed counters.

### Claim 40 (depends on 38)

The device of claim 38, wherein a usage of the closed counter is time-multiplexed and shared by multiple open counters.

### Claim 41 (depends on 22)

The device of claim 22, wherein the requests are non-posed write requests.
