# US 12,001,209 B2 · Barriers and synchronization for machine learning at autonomous machines

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,001,209 B2
- Document ID: US12001209B2
- Status: Granted
- Kind: B2
- Application number: 17/750,917
- Filed: 2022-05-23
- Published: 2022-11-10
- Granted: 2024-06-04
- Earliest claimed: 2017-04-24
- Assignees: Intel Corporation
- Applicants: Intel Corporation
- Inventors: Abhishek R. Appu; Altug Koker; Joydeep Ray; Balaji Vembu; John C. Weast; Mike B. Macpherson; Dukhwan Kim; Linda L. Hurd; Sanjeev Jahagirdar; Vasanth Ranganathan
- Web page: https://patentgrep.com/patent/US12001209B2

## CPC

- G06F 9/522: {Barrier synchronisation}
- G05D 1/00: Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots (drive control systems specially adapted for autonomous road vehicles B60W60/00)
- G05D 1/227: Handing over between remote control and on-board control; Handing over between remote control arrangements
- G06F 9/4881: {Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues}
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/063: using electronic means
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06F 9/46: Multiprogramming arrangements
- G06T 1/20: Processor architectures; Processor configuration, e.g. pipelining
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A method of embodiments, as described herein, includes detecting thread groups relating to machine learning associated with one or more processing devices. The method may further include facilitating barrier synchronization of the thread groups across multiple dies such that each thread in a thread group is scheduled across a set of compute elements associated with the multiple dies, where each die represents a processing device of the one or more processing devices, the processing device including a graphics processor.

## Claims

### Claim 1

A processing apparatus comprising:
a first processing resource;
a second processing resource coupled with the first processing resource, wherein the first processing resource and the second processing resource each include a separate respective local memory and instruction execution resources configured to concurrently execute a plurality of threads of a thread group;
a shared memory coupled with the first processing resource and the second processing resource, wherein the shared memory is accessible to the first processing resource and the second processing resource; and
scheduler hardware including first circuitry configured to:
receive the thread group for scheduling, wherein the thread group includes program code configured to store data within the local memory and is programmed for execution on a single processing resource;
based on a hint associated with the thread group, map the local memory to the shared memory;
schedule threads of the thread group for execution via multiple processing resources including the first processing resource and the second processing resource, wherein the program code of the thread group that is configured to store data within the local memory is to store the data within the shared memory;
second circuitry to:
determine that the thread group is associated with a hint;
evaluate a memory access performance impact of mapping the local memory to the shared memory based on an amount of shared memory to be accessed by the program code of the thread group due to the mapping; and
based on the evaluation, provide the hint to the first circuitry; and
wherein the first processing resource and the second processing resource are to execute scheduled threads of the thread group via respective instruction execution resources.

### Claim 2 (depends on 1)

The processing apparatus as in claim 1, wherein the hint includes a value determined based on an amount of data the program code is configured to store to the local memory.

### Claim 3 (depends on 2)

The processing apparatus as in claim 2, wherein the value indicates that the amount of data the program code is configured to store to the local memory is below a threshold.

### Claim 4 (depends on 1)

The processing apparatus as in claim 1, wherein the hint is stored to a thread group state that represents a state of the threads of the thread group.

### Claim 5 (depends on 4)

The processing apparatus as in claim 4, the second circuitry additionally configured to preempt execution of the thread group in response to a preemption event.

### Claim 6 (depends on 5)

The processing apparatus as in claim 5, wherein the preemption event includes a multi-die barrier event.

### Claim 7 (depends on 6)

The processing apparatus as in claim 6, wherein the multi-die barrier event is caused by execution of a multi-die barrier by a thread of the thread group.

### Claim 8 (depends on 7)

The processing apparatus as in claim 7, wherein the hint additionally includes an indication that the program code includes a barrier instruction associated with a local barrier that is limited to the first processing resource or the second processing resource.

### Claim 9 (depends on 8)

The processing apparatus as in claim 8, wherein, based on the indication, the first circuitry is to bypass the mapping of the local memory to the shared memory and schedule the thread group to the first processing resource or the second processing resource.

### Claim 10

A data processing system comprising:
a memory device; and
a graphics processor coupled with the memory device, the graphics processor including:
a first processing resource;
a second processing resource coupled with the first processing resource, wherein the first processing resource and the second processing resource each include a separate respective local memory and instruction execution resources configured to concurrently execute a plurality of threads;
a shared memory coupled with the first processing resource and the second processing resource, wherein the shared memory is accessible to the first processing resource and the second processing resource; and
scheduler hardware including first circuitry configured to:
receive a thread group for scheduling, wherein the thread group includes program code configured to store data within the local memory and is programmed for execution on a single processing resource;
based on a hint associated with the thread group, map the local memory to the shared memory; and
schedule threads of the thread group for execution via multiple processing resources including the first processing resource and the second processing resource, wherein the program code of the thread group that is configured to store data within the local memory is to store the data within the shared memory;
second circuitry to:
determine that the thread group is associated with a hint;
evaluate a memory access performance impact of mapping the local memory to the shared memory based on an amount of shared memory accessed by the program code of the thread group; and
based on the evaluation, provide the hint to the first circuitry; and
wherein the first processing resource and the second processing resource are to execute scheduled threads of the thread group via respective instruction execution resources.

### Claim 11 (depends on 10)

The data processing system as in claim 10, wherein the hint includes a value determined based on an amount of data the program code is configured to store to the local memory and the value indicates that the amount of data the program code is configured to store to the local memory is below a threshold.

### Claim 12 (depends on 10)

The data processing system as in claim 10, the second circuitry additionally configured to preempt execution of the thread group in response to a preemption event, wherein the preemption event includes a multi-die barrier event that is caused by execution of a multi-die barrier by a thread of the thread group.

### Claim 13 (depends on 12)

The data processing system as in claim 12, wherein the hint additionally includes an indication that the program code includes a barrier instruction associated with a local barrier that is limited to the first processing resource or the second processing resource and based on the indication, the first circuitry is to bypass the mapping of the local memory to the shared memory and schedule the thread group to the first processing resource or the second processing resource.

### Claim 14

A method comprising:
receiving a thread group for scheduling to multiple processing resources each including a respective local memory and instruction execution resources configured to concurrently execute a plurality of threads;
determining that the thread group is associated with a hint;
evaluating, at second circuitry, a memory access performance impact of mapping respective local memories of the multiple processing resources to a shared memory based on an amount of shared memory to be accessed by program code of the thread group due to the mapping;
providing the hint to first circuitry based on the evaluation;
mapping, at the first circuitry, the respective local memories of the multiple processing resources to the shared memory based on the hint provided by the second circuitry;
scheduling threads of the thread group for execution to the multiple processing resources, wherein the thread group is associated with program code that is configured to store data within respective local memories of the multiple processing resources;
executing the thread group via the multiple processing resources, wherein, within a processing resource of the multiple processing resources, and based on the mapping, the program code associated with the thread group is to store the data within the shared memory instead of the respective local memories of the multiple processing resources.

### Claim 15 (depends on 14)

The method as in claim 14, wherein the hint includes a value determined based on an amount of data the program code is configured to store to the local memory.

### Claim 16 (depends on 15)

The method as in claim 15, wherein the value indicates that the amount of data the program code is configured to store to the local memory is below a threshold.

### Claim 17 (depends on 14)

The method as in claim 14, wherein the hint is stored to a thread group state that represents a state of threads of the thread group.

### Claim 18 (depends on 17)

The method as in claim 17, the second circuitry additionally configured to preempt execution of the thread group in response to a preemption event, wherein the preemption event includes a multi-die barrier event caused by execution of a multi-die barrier by a thread of the thread group.

### Claim 19 (depends on 18)

The method as in claim 18, wherein the hint additionally includes an indication that the program code includes a barrier instruction associated with a local barrier that is limited to a single processing resource.

### Claim 20 (depends on 19)

The method as in claim 19, further comprising, based on the indication, bypassing the mapping of the respective local memories to the shared memory and scheduling the thread group to the single processing resource.