US 2026/0187186 A1Published application

Enabling parallel variable-rate compression and decompression of values of a tensor

AssigneeMicrosoft Technology Licensing, LLC

InventorLi ZHANG

Life of this patent

Estimating the term…

Loading patent…

Abstract

For purposes of compression and decompression operations, a tensor is split into tiles having a tile size, and values at a given position in the respective tiles are processed as a stream. In some example implementations, for compression operations, a matrix processing tool can separately apply variable-rate compression to different streams of values of the tensor, with the different streams being associated with different positions of the tile size. For decompression operations, the matrix processing tool can quickly reconstruct different streams of values of the tensor. The decompression operations can be performed in parallel, with different decoders decompressing different streams of values of the tensor. In this way, soon after decompression starts, tensor blocks of values of the tensor can be loaded to local memory of tensor cores for matrix multiplication and accumulation operations, which can help reduce latency and fully utilize the tensor cores.

Claims20

Dependent claims

A computer system comprising multiple processing cores, multiple tensor cores, and memory, the computer system being configured to perform operations comprising:

receiving values of a tensor, the values of the tensor being logically organized along multiple dimensions, the tensor being split into multiple tiles having a tile size with multiple positions, each of the multiple tiles including a different subset of the values of the tensor;
and
encoding a group of tiles among the multiple tiles as multiple streams, wherein different positions among the multiple positions of the tile size are associated with different streams among the multiple streams, and wherein the encoding the group of tiles includes, for each given position among the multiple positions of the tile size:
loading at least some of the values at the given position across the group of tiles, respectively, in a given stream among the multiple streams;
encoding the at least some of the values at the given position across the group of tiles, respectively, thereby producing encoded data for the at least some of the values at the given position across the group of tiles, respectively;
and
outputting the encoded data for the at least some of the values at the given position across the group of tiles, respectively.

4 more elements

In a computer system, a method comprising:

receiving encoded data for values of a tensor, the values of the tensor being logically organized along multiple dimensions, the tensor being split into multiple tiles having a tile size with multiple positions, each of the multiple tiles including a different subset of the values of the tensor;
decoding a group of tiles among the multiple tiles as multiple streams, wherein different positions among the multiple positions of the tile size are associated with different streams among the multiple streams, and wherein the decoding the group of tiles includes, for each given position among the multiple positions of the tile size:
loading encoded data for at least some of the values at the given position across the group of tiles, respectively;
decoding the encoded data for the at least some of the values at the given position across the group of tiles, respectively, thereby producing the at least some of the values at the given position across the group of tiles, respectively, in a given stream among the multiple streams;
and
outputting the at least some of the values at the given position across the group of tiles, respectively, in the given stream.

3 more elements

Dependencies are inferred from the claim text.