# US 2018/0322064 A1 · Memory allocation system for multi-tier memory

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0322064 A1
- Document ID: US20180322064A1
- Status: Published application
- Kind: A1
- Application number: 16/034,216
- Filed: 2018-07-12
- Published: 2018-11-08
- Earliest claimed: 2016-12-09
- Assignees: Cray Inc.
- Applicants: Cray, Inc.
- Inventors: Heidi Lynn Poxon; William Homer; David W. Oehmke; Luiz DeRose; Clayton D. Andreasen; Sanyam Mehta
- Web page: https://patentgrep.com/patent/US20180322064A1

## CPC

- G06F 12/0284: {Multiple user address space allocation, e.g. using different base addresses (interprocessor communication G06F15/163)}
- G06F 8/41: Compilation
- G06F 8/443: {Optimisation}
- G06F 11/3037: {where the computing system component is a memory, e.g. virtual memory, cache (accessing, addressing or allocating within memory systems or architectures G06F12/00; checking stores for correct operation G11C29/00)}
- G06F 11/3409: {for performance assessment}
- G06F 11/3452: {Performance evaluation by statistical analysis}
- G06F 11/3471: {Address tracing}
- G06F 12/023: {Free address space management}
- G06F 12/08: in hierarchically structured memory systems, e.g. virtual memory systems
- G06F 12/0871: Allocation or management of cache space
- G06F 2201/885: Monitoring specific for caches
- G06F 2212/1016: Performance improvement
- G06F 2212/1021: Hit rate improvement
- G06F 2212/465: Structured object, e.g. database record
- G06F 2212/601: Reconfiguration of cache memory
- G06F 2212/604: Details relating to cache allocation
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A system is provided for allocating memory for data of a program for execution by a computer system with a multi-tier memory that includes LBM and HBM. The system accesses a data structure map that maps data structures of the program to the memory addresses within an address space of the program to which the data structures are initially allocated. The system executes the program to collect statistics relating to memory requests and memory bandwidth utilization of the program. The system determines an extent to which each data structure is used by a high memory utilization portion of the program based on the data structure map and the collected statistics. The system generates a memory allocation plan that favors allocating data structures in HBM based on the extent to which the data structures are used by a high memory utilization portion of the program.

## Claims

### Claim 1

A method performed by a computer system for collecting execution statistics of a program, the method comprising:
during execution of the program,
specifying an event for collecting statistics upon occurrences of the event; and
upon an occurrence of the event,
retrieving an instruction address of a load instruction associated with the event;
retrieving a data address of data of the load instruction associated with the event;
collecting memory request statistics associated with the event;
collecting memory bandwidth utilization statistics associated with the event; and
associating with the occurrence of the event the instruction address, the data address, the memory request statistics, and the memory bandwidth utilization statistics as collected statistics for the occurrence of the event.

### Claim 2 (depends on 1)

The method of claim 1 further comprising during execution of the program:
accessing a data structure map that maps data structures of the program to the memory addresses within an address space of the program to which the data structures are allocated; and
associating the collected statistics for an occurrence of the event with the data structure whose memory addresses include the data address associated with the event.

### Claim 3 (depends on 1)

The method of claim 1 further comprising after execution of the program:
accessing a data structure map that maps data structures of the program to the memory addresses within an address space of the program to which the data structures are allocated; and
associating the collected statistics for an occurrence of the event with the data structure whose memory addresses include the data address associated with the event.

### Claim 4 (depends on 1)

The method of claim 1 wherein an event occurs when a threshold level of a performance counter is reached.

### Claim 5 (depends on 4)

The method of claim 4 wherein the performance counter is the number of last level cache misses since the last occurrence of the event.

### Claim 6 (depends on 1)

The method of claim 1 wherein the event occurs at time intervals.

### Claim 7 (depends on 1)

The method of claim 1 wherein the memory request statistics include the number of last level cache misses since the last occurrence of the event.

### Claim 8 (depends on 7)

The method of claim 7 wherein the memory request statistics further include the number of last level cache hits since the last occurrence of the event.

### Claim 9 (depends on 8)

The method of claim 8 wherein the memory bandwidth utilization statistics include the number of instructions executed since the last occurrence of the event.

### Claim 10 (depends on 9)

The method of claim 9 wherein the memory bandwidth utilization statistics include a time of the occurrence of the event.

### Claim 11 (depends on 9)

The method of claim 9 wherein the memory bandwidth utilization statistics include the number of processor cycles since the last occurrence of the event.

### Claim 12 (depends on 9)

The method of claim 9 wherein the memory bandwidth utilization statistics include the number of processor-stalled cycles since the last occurrence of the event.

### Claim 13 (depends on 1)

The method of claim 1 wherein the memory bandwidth utilization statistics include a time of the occurrence of the event.

### Claim 14 (depends on 1)

The method of claim 1 wherein the memory bandwidth utilization statistics include the number of processor cycles since the last occurrence of the event.

### Claim 15

A computing system for collecting execution statistics of a program, the computing system comprising:
one or more computer-readable storage mediums storing computer-executable instructions for controlling the computing system to: upon an occurrence of the event,
retrieve an instruction address of a load instruction associated with the event;
retrieve a data address of data of the load instruction associated with the event;
collect memory statistics associated with the event; and
associate with the occurrence of the event the instruction address, the data address, and the memory statistics, as collected statistics for the occurrence of the event; and
one or more processors for executing the computer-executable instructions stored in the one or more computer-readable storage mediums.

### Claim 16 (depends on 15)

The computing system of claim 15 wherein the computer-executable instructions further control the computing system to:
access a data structure map that maps data structures of the program to the memory addresses within an address space of the program to which the data structures are allocated; and
associate the collected memory statistics for an occurrence of the event with the data structure whose memory addresses include the data address associated with the event.

### Claim 17 (depends on 15)

The computing system of claim 15 wherein an event occurs when a threshold level of a performance counter is reached.

### Claim 18 (depends on 17)

The computing system of claim 17 wherein the performance counter is the number of last level cache misses since the last occurrence of the event.

### Claim 19 (depends on 15)

The computing system of claim 15 wherein the event occurs at time intervals.

### Claim 20 (depends on 15)

The computing system of claim 15 wherein the memory statistics include the number of last level cache misses since the last occurrence of the event.

### Claim 21 (depends on 15)

The computing system of claim 15 wherein the memory statistics include bandwidth utilization statistics.

### Claim 22 (depends on 15)

The computing system of claim 15 wherein the memory statistics include memory request statistics.

### Claim 23 (depends on 15)

The computing system of claim 15 wherein the memory statistics include bandwidth utilization statistics and memory request statistics.
