# US 2014/0195744 A1 · On-chip traffic prioritization in memory

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2014/0195744 A1
- Document ID: US20140195744A1
- Status: Published application
- Kind: A1
- Application number: 13/761,252
- Filed: 2013-02-07
- Published: 2014-07-10
- Earliest claimed: 2013-01-09
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: Bruce M. Fleischer; Thomas W. Fox; Hans M. Jacobson; Ravi Nair
- Web page: https://patentgrep.com/patent/US20140195744A1

## CPC

- G06F 3/0659: {Command handling arrangements, e.g. command buffers, queues, command scheduling}
- G06F 3/061: {Improving I/O performance}
- G06F 3/0625: {Power saving in storage systems}
- G06F 3/0673: {Single storage device}
- G06F 13/1626: {by reordering requests}
- G06F 13/1663: {Access to shared memory}
- G06F 13/18: based on priority control (G06F13/1605 takes precedence)
- G06F 9/4881: {Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues}
- Y02D 10/00: Energy efficient computing, e.g. low power processors, power management or thermal management
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- Y02D: CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE

## Abstract

According to one embodiment, a memory device is provided. The memory device includes a processing element coupled to a crossbar interconnect. The processing element is configured to send a memory access request, including a priority value, to the crossbar interconnect. The crossbar interconnect is configured to route the memory access request to a memory controller associated with the memory access request. The memory controller is coupled to memory and to the crossbar interconnect. The memory controller includes a queue and is configured to compare the priority value of the memory access request to priority values of a plurality of memory access requests stored in the queue of the memory controller to determine a highest priority memory access request and perform a next memory access request based on the highest priority memory access request.

## Claims

### Claim 1

A memory device, comprising:
a processing element coupled to a crossbar interconnect, the processing element configured to send a memory access request comprising a priority value to the crossbar interconnect;
the crossbar interconnect configured to route the memory access request to a memory controller associated with the memory access request; and
the memory controller coupled to memory and to the crossbar interconnect, the memory controller comprising a queue and configured to compare the priority value of the memory access request to priority values of a plurality of memory access requests stored in the queue of the memory controller to determine a highest priority memory access request and perform a next memory access request based on the highest priority memory access request.

### Claim 2 (depends on 1)

The memory device of claim 1, wherein the processing element further comprises a memory request priority register, and the processing element is further configured to set the priority value based on the memory request priority register in the processing element.

### Claim 3 (depends on 2)

The memory device of claim 2, wherein the memory request priority register is adjusted based on an application code hint indicating criticality of a code section.

### Claim 4 (depends on 2)

The memory device of claim 2, wherein the processing element is further configured to:
adjust the memory request priority register based on a load-to-use distance hint from a compiler;
handle the memory access request as a non-speculative request based on a lower priority value; and
handle the memory access request as a speculative request based on a higher priority value.

### Claim 5 (depends on 2)

The memory device of claim 2, wherein the processing element further comprises a load-store queue and the processing element is further configured to adjust the memory request priority register based on a load-store queue depth relative to a load-store queue capacity of the load-store queue.

### Claim 6 (depends on 5)

The memory device of claim 5, wherein the memory controller is further configured to:
receive an indication that the load-store queue has reached a threshold level of the load-store queue depth relative to the load-store queue capacity; and
increase priority values of memory access requests in the queue of the memory controller from the processing element based on the indication.

### Claim 7 (depends on 2)

The memory device of claim 2, wherein the processing element further comprises a progress counter and the processing element is further configured to:
update the progress counter in the processing element based on a relative number of instructions executed in the processing element; and
adjust the memory request priority register based on the progress counter.

### Claim 8 (depends on 7)

The memory device of claim 7, further comprising:
one or more other processing elements comprising one or more other memory request priority registers and one or more other progress counters; and
a progress monitor coupled to the processing element and the one or more other processing elements, the progress monitor configured to:
receive a progress counter value of the progress counter and one or more other progress counter values from the one or more other progress counters;
compare the progress counter value to the one or more other progress counter values; and
adjust the memory request priority register of the processing element and the one or more other memory request priority registers of the one or more other processing elements to increase priority for processing elements having lesser progress and decrease priority for processing elements having greater progress.

### Claim 9 (depends on 1)

The memory device of claim 1, wherein the crossbar interconnect further comprises an arbiter, the arbiter configured to:
receive the memory access request and one or more other memory access requests; and
arbitrate between the memory access request and the one or more other memory access requests based on comparing the priority value of the memory access request relative to one or more priority values of the one or more other memory access requests.

### Claim 10 (depends on 1)

The memory device of claim 1, wherein the memory controller is further configured to adjust the priority values of the plurality of memory access requests stored in the queue of the memory controller based on one or more of:
a number of memory access requests in the queue from a same processing element;
a latest priority of a last arriving memory access request from the same processing element; and
an age of the memory access requests in the queue.

### Claim 11

A memory system, comprising:
a processing element coupled to a crossbar interconnect of a memory device and one or more other processing elements coupled to the crossbar interconnect of the memory device, the processing element configured to send a memory access request comprising a priority value to the crossbar interconnect; and
the memory device comprising:
a plurality of memory controllers;
a vault of stacked memory coupled to each of the plurality of memory controllers;
the crossbar interconnect configured to route the memory access request to a memory controller of the plurality of memory controllers associated with the memory access request; and
the memory controller associated with the memory access request comprising a queue and configured to compare the priority value of the memory access request to priority values of a plurality of memory access requests stored in the queue of the memory controller to determine a highest priority memory access request and perform a next memory access request based on the highest priority memory access request, the plurality of memory access requests from the processing element and the one or more other processing elements.

### Claim 12 (depends on 11)

The memory system of claim 11, wherein the processing element further comprises a memory request priority register, and the processing element is further configured to set the priority value based on the memory request priority register in the processing element.

### Claim 13 (depends on 12)

The memory system of claim 12, wherein the memory request priority register is adjusted based on an application code hint indicating criticality of a code section.

### Claim 14 (depends on 12)

The memory system of claim 12, further comprising a compiler configured to determine a load-to-use distance between loading and using data associated with the memory access request when generating executable instructions for the processing element and to set a load-to-use distance hint in the executable instructions for the processing element, wherein the processing element is further configured to:
adjust the memory request priority register based on the load-to-use distance hint from the compiler, wherein a larger load-to-use distance is associated with a higher priority and a smaller load-to-use distance is associated with a lower priority;
handle the memory access request as a non-speculative request based on a lower priority value; and
handle the memory access request as a speculative request based on a higher priority value.

### Claim 15 (depends on 12)

The memory system of claim 12, wherein the processing element further comprises a load-store queue and the processing element is further configured to adjust the memory request priority register based on a load-store queue depth relative to a load-store queue capacity of the load-store queue.

### Claim 16 (depends on 15)

The memory system of claim 15, wherein the memory controller associated with the memory access request is further configured to:
receive an indication that the load-store queue has reached a threshold level of the load-store queue depth relative to the load-store queue capacity; and
increase priority values of memory access requests in the queue of the memory controller from the processing element based on the indication.

### Claim 17 (depends on 12)

The memory system of claim 12, wherein the processing element further comprises a progress counter and the processing element is further configured to:
update the progress counter in the processing element based on a relative number of instructions executed in the processing element; and
adjust the memory request priority register based on the progress counter.

### Claim 18 (depends on 17)

The memory system of claim 17, wherein the one or more other processing elements comprise one or more other memory request priority registers and one or more other progress counters; and further comprising:
a progress monitor coupled to the processing element and the one or more other processing elements, the progress monitor configured to:
receive a progress counter value of the progress counter and one or more other progress counter values from the one or more other progress counters;
compare the progress counter value to the one or more other progress counter values; and
adjust the memory request priority register of the processing element and the one or more other memory request priority registers of the one or more other processing elements to increase priority for processing elements having lesser progress and decrease priority for processing elements having greater progress.

### Claim 19 (depends on 11)

The memory system of claim 11, wherein the crossbar interconnect further comprises an arbiter, the arbiter configured to:
receive the memory access request and one or more other memory access requests; and
arbitrate between the memory access request and the one or more other memory access requests based on comparing the priority value of the memory access request relative to one or more priority values of the one or more other memory access requests.

### Claim 20 (depends on 11)

The memory system of claim 11, wherein the memory controller associated with the memory access request is further configured to adjust the priority values of the plurality of memory access requests stored in the queue of the memory controller based on one or more of:
a number of memory access requests in the queue from a same processing element;
a latest priority of a last arriving memory access request from the same processing element; and
an age of the memory access requests in the queue.