# US 2014/0195743 A1 · On-chip traffic prioritization in memory

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2014/0195743 A1
- Document ID: US20140195743A1
- Status: Published application
- Kind: A1
- Application number: 13/737,339
- Filed: 2013-01-09
- Published: 2014-07-10
- Earliest claimed: 2013-01-09
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: Bruce M. Fleischer; Thomas W. Fox; Hans M. Jacobson; Ravi Nair
- Web page: https://patentgrep.com/patent/US20140195743A1

## CPC

- G06F 3/0659: {Command handling arrangements, e.g. command buffers, queues, command scheduling}
- G06F 3/061: {Improving I/O performance}
- G06F 3/0625: {Power saving in storage systems}
- G06F 3/0673: {Single storage device}
- G06F 13/1626: {by reordering requests}
- G06F 13/1663: {Access to shared memory}
- G06F 13/18: based on priority control (G06F13/1605 takes precedence)
- G06F 9/4881: {Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues}
- Y02D 10/00: Energy efficient computing, e.g. low power processors, power management or thermal management
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- Y02D: CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE

## Abstract

According to one embodiment, a method for traffic prioritization in a memory device includes sending a memory access request including a priority value from a processing element in the memory device to a crossbar interconnect in the memory device. The memory access request is routed through the crossbar interconnect to a memory controller in the memory device associated with the memory access request. The memory access request is received at the memory controller. The priority value of the memory access request is compared to priority values of a plurality of memory access requests stored in a queue of the memory controller to determine a highest priority memory access request. A next memory access request is performed by the memory controller based on the highest priority memory access request.

## Claims

### Claim 1

A method for traffic prioritization in a memory device, the method comprising:
sending a memory access request comprising a priority value from a processing element in the memory device to a crossbar interconnect in the memory device;
routing the memory access request through the crossbar interconnect to a memory controller in the memory device associated with the memory access request;
receiving the memory access request at the memory controller;
comparing the priority value of the memory access request to priority values of a plurality of memory access requests stored in a queue of the memory controller to determine a highest priority memory access request; and
performing a next memory access request by the memory controller based on the highest priority memory access request.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
setting the priority value based on a memory request priority register in the processing element.

### Claim 3 (depends on 2)

The method of claim 2, further comprising:
adjusting the memory request priority register based on an application code hint indicating criticality of a code section.

### Claim 4 (depends on 2)

The method of claim 2, further comprising:
adjusting the memory request priority register based on a load-to-use distance hint from a compiler;
handling the memory access request as a non-speculative request based on a lower priority value; and
handling the memory access request as a speculative request based on a higher priority value.

### Claim 5 (depends on 2)

The method of claim 2, further comprising:
adjusting the memory request priority register based on a load-store queue depth relative to a load-store queue capacity of a load-store queue in the processing element.

### Claim 6 (depends on 5)

The method of claim 5, further comprising:
receiving at the memory controller an indication that the load-store queue has reached a threshold level of the load-store queue depth relative to the load-store queue capacity; and
increasing priority values of memory access requests in the queue of the memory controller from the processing element based on the indication.

### Claim 7 (depends on 2)

The method of claim 2, further comprising:
updating a progress counter in the processing element based on a relative number of instructions executed in the processing element; and
adjusting the memory request priority register based on the progress counter.

### Claim 8 (depends on 7)

The method of claim 7, further comprising:
receiving a progress counter value of the progress counter at a progress monitor;
comparing the progress counter value to one or more other progress counter values received from one or more other processing elements; and
adjusting the memory request priority register of the processing element and one or more other memory request priority registers of the one or more other processing elements to increase priority for processing elements having lesser progress and decrease priority for processing elements having greater progress.

### Claim 9 (depends on 1)

The method of claim 1, further comprising:
receiving the memory access request and one or more other memory access requests at the crossbar interconnect; and
arbitrating between the memory access request and the one or more other memory access requests in the crossbar interconnect based on comparing the priority value of the memory access request relative to one or more priority values of the one or more other memory access requests.

### Claim 10 (depends on 1)

The method of claim 1, further comprising:
adjusting the priority values of the plurality of memory access requests stored in the queue of the memory controller based on one or more of:
a number of memory access requests in the queue from a same processing element;
a latest priority of a last arriving memory access request from the same processing element; and
an age of the memory access requests in the queue.

### Claim at position 11

11-20. (canceled)

### Claim 21

A method for traffic prioritization in a memory system, the method comprising:
sending a memory access request comprising a priority value from a processing element to a crossbar interconnect in a memory device;
routing the memory access request through the crossbar interconnect to a memory controller in the memory device associated with the memory access request;
receiving the memory access request at the memory controller to access memory in a vault of stacked memory chips controlled by the memory controller;
comparing the priority value of the memory access request to priority values of a plurality of memory access requests stored in a queue of the memory controller to determine a highest priority memory access request, the plurality of memory access requests from the processing element and one or more other processing elements; and
performing a next memory access request by the memory controller based on the highest priority memory access request.

### Claim 22 (depends on 21)

The method of claim 21, further comprising:
setting the priority value based on a memory request priority register in the processing element.

### Claim 23 (depends on 22)

The method of claim 22, further comprising:
adjusting the memory request priority register based on an application code hint indicating criticality of a code section.

### Claim 24 (depends on 22)

The method of claim 22, further comprising:
determining a load-to-use distance between loading and using data associated with the memory access request by a compiler when generating executable instructions for the processing element;
setting a load-to-use distance hint in the executable instructions for the processing element by the compiler;
adjusting the memory request priority register based on the load-to-use distance hint from the compiler, wherein a larger load-to-use distance is associated with a higher priority and a smaller load-to-use distance is associated with a lower priority;
handling the memory access request as a non-speculative request based on a lower priority value; and
handling the memory access request as a speculative request based on a higher priority value.

### Claim 25 (depends on 22)

The method of claim 22, further comprising:
adjusting the memory request priority register based on a load-store queue depth relative to a load-store queue capacity of a load-store queue in the processing element.

### Claim 26 (depends on 25)

The method of claim 25, further comprising:
receiving at the memory controller an indication that the load-store queue has reached a threshold level of the load-store queue depth relative to the load-store queue capacity; and
increasing priority values of memory access requests in the queue of the memory controller from the processing element based on the indication.

### Claim 27 (depends on 22)

The method of claim 22, further comprising:
updating a progress counter in the processing element based on a relative number of instructions executed in the processing element; and
adjusting the memory request priority register based on the progress counter.

### Claim 28 (depends on 27)

The method of claim 27, further comprising:
receiving a progress counter value of the progress counter at a progress monitor;
receiving one or more other progress counter values from the one or more other processing elements;
comparing the progress counter value to the one or more other progress counter values; and
adjusting the memory request priority register of the processing element and one or more other memory request priority registers of the one or more other processing elements to increase priority for processing elements having lesser progress and decrease priority for processing elements having greater progress.

### Claim 29 (depends on 21)

The method of claim 21, further comprising:
receiving the memory access request and one or more other memory access requests at the crossbar interconnect; and
arbitrating between the memory access request and the one or more other memory access requests in the crossbar interconnect based on comparing the priority value of the memory access request relative to one or more priority values of the one or more other memory access requests.

### Claim 30 (depends on 21)

The method of claim 21, further comprising:
adjusting the priority values of the plurality of memory access requests stored in the queue of the memory controller based on one or more of:
a number of memory access requests in the queue from a same processing element;
a latest priority of a last arriving memory access request from the same processing element; and
an age of the memory access requests in the queue.