# US 11,909,422 B2 · Neural network processor using compression and decompression of activation data to reduce memory bandwidth utilization

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,909,422 B2
- Document ID: US11909422B2
- Status: Granted
- Kind: B2
- Application number: 18/054,851
- Filed: 2022-11-11
- Published: 2023-03-09
- Granted: 2024-02-20
- Earliest claimed: 2017-04-17
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Joseph Leon Corkery; Benjamin Eliot Lundell; Larry Marvin Wall; Chad Balling McBride; Amol Ashok Ambardekar; George Petre; Kent D. Cedola; Boris Bobrov
- Web page: https://patentgrep.com/patent/US11909422B2

## CPC

- H03M 7/3059: {Digital compression and data reduction techniques where the original information is represented by a subset or similar information, e.g. lossy compression}
- G06F 1/3206: Monitoring of events, devices or parameters that trigger a change in power modality
- G06F 1/324: by lowering clock frequency
- G06F 1/3275: {Power saving in memory, e.g. RAM, cache}
- G06F 1/3287: by switching off individual functional units in the computer system
- G06F 3/0604: {Improving or facilitating administration, e.g. storage management}
- G06F 3/0631: {by allocating resources to storage systems}
- G06F 3/067: {Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]}
- G06F 9/30087: {Synchronisation or serialisation instructions}
- G06F 9/3836: {Instruction issuing, e.g. dynamic instruction scheduling or out of order instruction execution}
- G06F 9/3858: {Result writeback, i.e. updating the architectural state or memory}
- G06F 9/3887: {controlled by a single instruction for multiple data lanes [SIMD]}
- G06F 9/46: Multiprogramming arrangements
- G06F 9/4881: {Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues}
- G06F 9/5016: {the resource being the memory}
- G06F 9/5033: {considering data affinity}
- G06F 9/5061: {Partitioning or combining of resources}
- G06F 9/5077: {Logical partitioning of resources; Management or configuration of virtualized resources (specific details on emulation or internal functioning of virtual machines G06F9/455)}
- G06F 12/0207: {with multidimensional access, e.g. row/column, matrix}
- G06F 12/0238: {Memory management in non-volatile memory, e.g. resistive RAM or ferroelectric memory}
- G06F 12/08: in hierarchically structured memory systems, e.g. virtual memory systems
- G06F 12/0862: with prefetch
- G06F 12/10: Address translation
- G06F 13/1673: {using buffers}
- G06F 13/1689: {Synchronisation and timing concerns (synchronisation on a memory bus G06F13/4234)}
- G06F 13/28: using burst mode transfer, e.g. direct memory access {DMA}, cycle steal (G06F13/32 takes precedence)
- G06F 15/8007: {single instruction multiple data [SIMD] multiprocessors}
- G06F 17/15: Correlation function computation {including computation of convolution operations (arithmetic circuits for sum of products per se, e.g. multiply-accumulators G06F7/5443; digital filters, e.g. FIR, IIR, adaptive filters H03H17/00)}
- G06N 3/045: Combinations of networks
- G06N 3/049: Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
- G06N 3/06: Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
- G06N 3/063: using electronic means
- G06N 3/065: Analogue means
- G06N 3/10: Interfaces, programming languages or software development kits, e.g. for simulating neural networks
- H03M 7/3066: {by means of a mask or a bit-map}
- H03M 7/6005: {Decoder aspects}
- H03M 7/6011: {Encoder aspects}
- H03M 7/6058: {Saving memory space in the encoder or decoder}
- H03M 7/70: {Type of the data to be coded, other than image and sound}
- H04L 45/04: {Interdomain routing, e.g. hierarchical routing}
- H04L 67/02: based on web technology, e.g. hypertext transfer protocol [HTTP]
- H04L 67/1001: for accessing one among a plurality of replicated servers
- G06F 9/467: {Transactional memory (G06F9/528 takes precedence)}
- G06F 2209/484: Precedence
- G06F 2209/485: Resource constraint
- G06F 2212/1016: Performance improvement
- G06F 2212/6026: Prefetching based on access pattern detection, e.g. stride based prefetch
- G06F 2212/657: Virtual address space management
- H03M 7/46: Conversion to or from run-length codes, i.e. by representing the number of consecutive digits, or groups of digits, of the same kind by a code word and a digit indicative of that kind
- H04L 45/50: using label swapping, e.g. multi-protocol label switch [MPLS]
- Y02D 10/00: Energy efficient computing, e.g. low power processors, power management or thermal management
- Y02D 30/50: in wire-line communication networks, e.g. low power modes or reduced link rate
- H03M: CODING; DECODING; CODE CONVERSION IN GENERAL (using fluidic means F15C4/00; optical analogue/digital converters G02F7/00; coding, decoding or code conversion, specially adapted for particular applications, see the relevant subclasses, e.g. G01D, G01R, G06F, G06T, G09G, G10L, G11B, G11C, H04B, H04L, H04M, H04N; ciphering or deciphering for cryptography or other purposes involving the need for secrecy G09C)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- Y02D: CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE

## Abstract

A deep neural network (“DNN”) module compresses and decompresses neuron-generated activation data to reduce the utilization of memory bus bandwidth. The compression unit receives an uncompressed chunk of data generated by a neuron in the DNN module. The compression unit generates a mask portion and a data portion of a compressed output chunk. The mask portion encodes the presence and location of the zero and non-zero bytes in the uncompressed chunk of data. The data portion stores truncated non-zero bytes from the uncompressed chunk of data. A decompression unit receives a compressed chunk of data from memory in the DNN processor or memory of an application host. The decompression unit decompresses the compressed chunk of data using the mask portion and the data portion.

## Claims

### Claim 1

A computer-implemented method, comprising:
receiving a compressed chunk of data comprising a mask portion and a data portion;
determining a number of non-zero bytes in a decompressed chunk of data based upon bits in the mask portion of the compressed chunk of data;
determining, based at least in part on the number of non-zero bytes in the decompressed chunk of data, a number of bits used to store truncated non-zero bytes in the data portion of the compressed chunk of data;
for each bit position in the mask portion of the compressed chunk of data that is a logical zero, inserting a zero byte into a corresponding position of the decompressed chunk of data; and
for each position in the mask portion of the compressed chunk of data that is a logical one, inserting a truncated non-zero byte from the corresponding position in the data portion of the compressed chunk of data into a corresponding position in the decompressed chunk of data and a number of zero bits equivalent to a number of bits truncated during compression of the compressed chunk of data.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, further comprising:
determining a number of additional bits allocated to the one or more of the non-zero bytes stored in the data portion of the compressed output chunk; and
assigning the determined number of additional bits to one or more of the non-zero bytes in the decompressed chunk of data.

### Claim 3 (depends on 1)

The computer-implemented method of claim 1, further comprising adding an offset to one or more of the truncated non-zero bytes stored in the decompressed output chunk.

### Claim 4 (depends on 1)

The computer-implemented method of claim 1, wherein one or more least significant bits (LSBs) of the non-zero bytes are truncated.

### Claim 5 (depends on 1)

The computer-implemented method of claim 1, wherein one or more most significant bits (MSBs) of the non-zero bytes are truncated.