Sparsity-aware datastore for inference processing in deep neural network architectures
AssigneeIntel Corporation
InventorsDeepak Mathaikutty, Arnab Raha, Raymond Sung, Debabrata Mohapatra, Cormac Brick
Life of this patent
Estimating the term…
Abstract
Systems, apparatuses and methods may provide for technology that prefetches compressed data and a sparsity bitmap from a memory to store the compressed data in a decode buffer, where the compressed data is associated with a plurality of tensors, wherein the compressed data is in a compressed format. The technology aligns the compressed data with the sparsity bitmap to generate decoded data, and provides the decoded data to a plurality of processing elements.
Claims25
A computing system comprising:
- a plurality of processing elements;
- a memory to store compressed data associated with a plurality of tensors and a sparsity bitmap, wherein the compressed data is to be in a compressed format;
- and
- a decode buffer that includes logic implemented at least partly in one or more of configurable or fixed-functionality hardware, the logic to:
- prefetch the compressed data from the memory;
- align the compressed data with the sparsity bitmap to generate decoded data;
- and
- provide the decoded data to the plurality of processing elements.
A semiconductor apparatus comprising:
- one or more substrates;
- and
- logic coupled to the one or more substrates, wherein the logic is implemented in one or more of configurable or fixed-functionality hardware, the logic to:
- prefetch compressed data and a sparsity bitmap from a memory to store the compressed data in a decode buffer, wherein the compressed data is associated with a plurality of tensors, wherein the compressed data is to be in a compressed format;
- align the compressed data with the sparsity bitmap to generate decoded data;
- and
- provide the decoded data to a plurality of processing elements.
At least one computer readable storage medium comprising a set of executable program instructions, which when executed by a computing system, cause the computing system to:
- prefetch compressed data and a sparsity bitmap from a memory to store the compressed data in a decode buffer, wherein the compressed data is associated with a plurality of tensors, wherein the data is to be in a compressed format;
- align the compressed data with the sparsity bitmap to generate decoded data;
- and
- provide the decoded data to a plurality of processing elements.
A method comprising:
- prefetching compressed data and a sparsity bitmap from a memory to store the compressed data in a decode buffer, wherein the compressed data is associated with a plurality of tensors, wherein the compressed data is in a compressed format;
- aligning the compressed data with the sparsity bitmap to generate decoded data;
- and
- providing the decoded data to a plurality of processing elements.
Dependencies are inferred from the claim text.