Direct memory access operation for neural network accelerator
AssigneeAmazon Technologies, Inc.
InventorsIlya Minkin, Ron Diamant, Kun Xu
Life of this patent
Estimating the term…
Abstract
In one example, an apparatus comprises: a direct memory access (DMA) descriptor queue that stores DMA descriptors, each DMA descriptor including an indirect address; an address translation table that stores an address mapping between indirect addresses and physical addresses; and a DMA engine configured to: fetch a DMA descriptor from the DMA descriptor queue to the address translation table to translate a first indirect address of the DMA descriptor to a first physical address based on the address mapping, and perform a DMA operation based on executing the DMA descriptor to transfer data to or from the first physical address.
Claims20
A neural network accelerator, comprising:
- a local memory;
- a programmable address translation table;
- an address mapping direct memory access (DMA) queue to store address mapping direct memory access (DMA) descriptors, each of the address mapping DMA descriptors including, as a source address, a physical address of address mapping data between physical addresses of a host memory and indirect addresses, each of the address mapping DMA descriptors further including, as a destination address, a physical address of the programmable address translation table;
- an input data DMA descriptor queue configured to store input data DMA descriptors, each input data DMA descriptor including an indirect address as a source address;
- an output data DMA descriptor queue configured to store output data DMA descriptors, each output data DMA descriptor including an indirect address as a destination address;
- a DMA engine;
- and
- a hardware processor configured to execute instructions associated with a neural network operation to:
- control the DMA engine to fetch a first address mapping DMA descriptor from the address mapping DMA queue, and to fetch a first address mapping to the programmable address translation table based on executing the first address mapping DMA descriptor;
- program the programmable address translation table based on the first address mapping;
- control the DMA engine to fetch a subset of the input data DMA descriptors from the input data DMA descriptor queue, and to translate first indirect addresses of the subset of the input data DMA descriptors to first physical addresses of the host memory via the programmable address translation table and based on the first address mapping;
- control the DMA engine to transfer input data from the first physical addresses of the host memory to the local memory based on executing the subset of the input data DMA descriptors;
- perform the neural network operation based on the input data from the local memory to generate output data, and to store the output data at the local memory;
- control the DMA engine to fetch a subset of the output data DMA descriptors from the output data DMA descriptor queue, and to translate second indirect addresses of the subset of the output data DMA descriptors to second physical addresses of the host memory via the programmable address translation table and based on the first address mapping;
- and
- control the DMA engine to transfer the output data from the local memory to the second physical addresses of the host memory based on executing the subset of the output data DMA descriptors.
An apparatus comprising:
- a local memory;
- a direct memory access (DMA) descriptor queue that stores DMA descriptors, each DMA descriptor including an indirect address;
- an address translation table that stores an address mapping between indirect addresses and physical addresses;
- a hardware processor coupled with the local memory and configured to execute a computation operation;
- and
- a DMA engine configured to:
- fetch a DMA descriptor from the DMA descriptor queue;
- translate, using the address translation table and based on the address mapping, a first indirect address of the DMA descriptor to a first physical address of a host memory;
- and
- perform a DMA operation based on executing the DMA descriptor to transfer input data from the first physical address of the host memory to the local memory to be operated on by the hardware processor during the computation operation.
A method, comprising:
- fetching a direct memory access (DMA) descriptor from a DMA descriptor queue that stores DMA descriptors, the DMA descriptor including a first indirect address;
- translating, using an address translation table that maps indirect addresses to physical addresses, the first indirect address to a first physical address of a host memory;
- and
- performing a DMA operation based on executing the DMA descriptor to transfer input data from the first physical address of the host memory to a local memory to be operated on by a hardware processor during a computation operation, the hardware processor coupled to the local memory.
Dependencies are inferred from the claim text.