Back to results
US 10,095,526 B2Granted
Technique for improving performance in multi-threaded processing units
AssigneeNVIDIA Corporation
InventorsSamuel H. Duncan, Gary Ward, M. Wasiur Rashid, Lincoln G. Garlick, Wojciech Jan Truty
Life of this patent
Estimating the term…
Abstract
A multi-threaded processing unit includes a hardware pre-processor coupled to one or more processing engines (e.g., copy engines, GPCs, etc.) that implement pre-emption techniques by dividing tasks into smaller subtasks and scheduling subtasks on the processing engines based on the priority of the tasks. By limiting the size of the subtasks, higher priority tasks may be executed quickly without switching the context state of the processing engine. Tasks may be subdivided based on a threshold size or by taking into account other consideration such as physical boundaries of the memory system.
Claims20
A computer-implemented method for improving performance of a multi-threaded processor, the method comprising:
- receiving a first task that is to be executed by a first processing engine that is implemented separately from a processing cluster array also included in the multi-threaded processor;
- dividing, by a pre-processor also included in the multi-threaded processor, the first task into a first set of subtasks, wherein each subtask in the first set of subtasks is smaller than a threshold size and is to be executed by the first processing engine;
- issuing at least one subtask from the first set of subtasks to the first processing engine for processing;
- receiving a second task that also is to be executed by the first processing engine, wherein the second task has a higher priority than the first task;
- dividing, by the pre-processor, the second task into a second set of subtasks, wherein each subtask in the second set of subtasks is smaller than the threshold size and is to be executed by the first processing engine;
- preempting the first task by issuing at least one subtask from the second set of subtasks to the first processing engine for processing prior to issuing all of the subtasks from the first set of subtasks to the first processing engine for processing;
- and
- issuing at least one additional subtask from the first set of subtasks to the first processing engine for processing after issuing the at least one subtask from the second set of subtasks to the first processing engine for processing.
5 more elements
A computer-implemented method for improving performance of a multi-threaded processor, the method comprising:
- receiving a first task that is to be executed by a first processing engine included in the multi-threaded processor, wherein the first task comprises a copy operation and the processing engine is a copy engine that is implemented separately from a processing cluster array also included the multi-threaded processor;
- dividing, by a pre-processor also included in the multi-threaded processor, the first task into a first set of subtasks, wherein each subtask in the first set of subtasks is smaller than a threshold size and is to be executed by the first processing engine;
- issuing at least one subtask from the first set of subtasks to the first processing engine for processing;
- receiving a second task that also is to be executed by the first processing engine, wherein the second task has a higher priority than the first task;
- dividing, by the pre-processor, the second task into a second set of subtasks, wherein each subtask in the second set of subtasks is smaller than the threshold size and is to be executed by the first processing engine;
- preempting the first task by issuing at least one subtask from the second set of subtasks to the first processing engine for processing prior to issuing all of the subtasks from the first set of subtasks to the first processing engine for processing;
- and
- issuing at least one additional subtask from the first set of subtasks to the first processing engine for processing after issuing the at least one subtask from the second set of subtasks to the first processing engine for processing.
5 more elements
A multi-threaded processor, comprising:
- a processing cluster array;
- one or more processing engines configured to execute tasks;
- and
- a pre-processor coupled to the one or more processing engines and configured to:
- receive a first task that is to be executed by a first processing engine that is implemented separately from the processing cluster array;
- divide the first task into a first set of subtasks, wherein each subtask in the first set of subtasks is smaller than a threshold size and is to be executed by the first processing engine;
- issue at least one subtask from the first set of subtasks to the first processing engine for processing;
- receive a second task that also is to be executed by the first processing engine, wherein the second task has a higher priority than the first task;
- divide the second task into a second set of subtasks, wherein each subtask in the second set of subtasks is smaller than the threshold size and is to be executed by the first processing engine;
- preempt the first task by issuing at least one subtask from the second set of subtasks to the first processing engine for processing prior to issuing all of the subtasks from the first set of subtasks to the first processing engine for processing;
- and
- issue at least one additional subtask from the first set of subtasks to the first processing engine for processing after issuing the at least one subtask from the second set of subtasks to the first processing engine for processing.
9 more elements
A multi-threaded processor, comprising:
- a processing cluster array;
- one or more processing engines configured to execute tasks;
- and
- a pre-processor coupled to the one or more processing engines and configured to:
- receive a first task that is to be executed by a first processing engine;
- divide the first task into a first set of subtasks, wherein each subtask in the first set of subtasks is smaller than a threshold size and is to be executed by the first processing engine, wherein the first task comprises a copy operation and the processing engine is a copy engine that is implemented separately from the processing cluster array;
- issue at least one subtask from the first set of subtasks to the first processing engine for processing;
- receive a second task that also is to be executed by the first processing engine, wherein the second task has a higher priority than the first task;
- divide the second task into a second set of subtasks, wherein each subtask in the second set of subtasks is smaller than the threshold size and is to be executed by the first processing engine;
- preempt the first task by issuing at least one subtask from the second set of subtasks to the first processing engine for processing prior to issuing all of the subtasks from the first set of subtasks to the first processing engine for processing;
- and
- issue at least one additional subtask from the first set of subtasks to the first processing engine for processing after issuing the at least one subtask from the second set of subtasks to the first processing engine for processing.
9 more elements
A computing device, comprising:
- a system memory;
- and
- a multi-threaded processor coupled to the system memory and including:
- a processing cluster array;
- one or more processing engines configured to execute tasks, and
- a pre-processor coupled to the one or more processing engines and configured to:
- receive a first task that is to be executed by a first processing engine that is implemented separately from the processing cluster array;
- divide the first task into a first set of subtasks, wherein each subtask in the first set of subtasks is smaller than a threshold size and is to be executed by the first processing engine;
- issue at least one subtask from the first set of subtasks to the first processing engine for processing;
- receive a second task that also is to be executed by the first processing engine, wherein the second task has a higher priority than the first task;
- divide the second task into a second set of subtasks, wherein each subtask in the second set of subtasks is smaller than the threshold size and is to be executed by the first processing engine;
- preempt the first task by issuing at least one subtask from the second set of subtasks to the first processing engine for processing prior to issuing all of the subtasks from the first set of subtasks to the first processing engine for processing;
- and
- issue at least one additional subtask from the first set of subtasks to the first processing engine for processing after issuing the at least one subtask from the second set of subtasks to the first processing engine for processing.
11 more elements
Dependencies are inferred from the claim text.