US 12,602,244 B2Granted

Offloading processing tasks to decoupled accelerators for increasing performance in a system on a chip

AssigneeNVIDIA Corporation

InventorsRavi P Singh, Ching-Yu Hung, Jagadeesh Sankaran, Ahmad Itani, Yen-Te Shih

Life of this patent

Estimating the term…

Loading patent…

Abstract

In various examples, a VPU and associated components may be optimized to improve VPU performance and throughput. For example, the VPU may include a min/max collector, automatic store predication functionality, a SIMD data path organization that allows for inter-lane sharing, a transposed load/store with stride parameter functionality, a load with permute and zero insertion functionality, hardware, logic, and memory layout functionality to allow for two point and two by two point lookups, and per memory bank load caching capabilities. In addition, decoupled accelerators may be used to offload VPU processing tasks to increase throughput and performance, and a hardware sequencer may be included in a DMA system to reduce programming complexity of the VPU and the DMA system. The DMA and VPU may execute a VPU configuration mode that allows the VPU and DMA to operate without a processing controller for performing dynamic region based data movement operations.

Claims20

Dependent claims

A system comprising:

a shared memory;
a processor coupled to the shared memory using a first communication interface, the processor to store, using the first communication interface and in the shared memory, at least:
a first task message indicating at least a first processing task for an accelerator to perform and an address within the shared memory for accessing a second task message;
the second task message indicating at least a second processing task for the accelerator to perform;
and
configuration information for performing first processing task and performing the second processing task;
the accelerator coupled to the shared memory using a second communication interface, the accelerator to:
access, using the second communication interface, the shared memory to obtain the first task message stored in the shared memory by the processor;
access, using the second communication interface and based at least first task message, at least a first portion of the configuration information stored in the shared memory to perform the first processing task;
after performing the first processing task, access, using the second communication interface and based at least on the address indicated by the first task message, the second task message stored in the shared memory by the processor;
and
access, using the second communication interface and based at least on the second task message, at least a second portion of the configuration information stored in the shared memory to perform the second processing task.

9 more elements

A decoupled accelerator comprising processing circuitry to:

read, from a vector memory (VMEM), a first task message generated and stored in the VMEM by a processor, the first task message indicating at least a first processing task to be performed and an address for accessing a second task message;
access, based at least on the first task message, a first portion of configuration information stored in the VMEM by the processor;
perform the first processing task using at least the first portion of the configuration information;
read, the VMEM and based at least on the address indicated by the first task message, a second task message generated and stored in the VMEM by the processor, the second task message indicating at least a second processing task to be performed;
access, based at least on the second task message, a second portion of the configuration information stored in the VMEM by the processor;
and
perform the second processing task using at least the second portion of the configuration information.

4 more elements

A method comprising:

reading, by an accelerator and from a memory, a first task message generated and stored in the memory by a processor, the first task message indicating at least a first processing task to be performed and an address for accessing a second task message;
accessing, by the accelerator and based at least on the first task message, a first portion of configuration information stored in the memory by the processor;
performing, by the accelerator, the first processing task using at least the first portion of the configuration information;
reading, by the accelerator and from the memory using the address indicated by the first task message, the second task message generated and stored in the memory by the processor, the second task message indicating at least a second processing task to be performed;
accessing, by the accelerator and based at least on the second task message, a second portion of the configuration information stored in the memory by the processor;
and
performing, by the accelerator, the second processing task using at least the second portion of the configuration information.

4 more elements

Dependencies are inferred from the claim text.

Facts

Application: 17/391,320Filed 2 Aug 2021
Published: US 2023/0042858 A19 Feb 2023
Granted: 14 Apr 2026
Term ends: Not estimatedThe source records don’t support an estimate.

Classification7 codes

G06F 9/3867
Concurrent instruction execution, e.g. pipeline or look ahead: using instruction pipelines
G06F 9/4843
Task transfer initiation or dispatching: by program, e.g. task dispatcher, supervisor, operating system
G06F 9/30098
Arrangements for executing machine instructions, e.g. instruction decode: register arrangements
G06F 9/30101
Register arrangements: special purpose registers