Compressing and Decompressing Data for Language Models

A method comprising:

• receiving, at a compression block included in a neural network, an input matrix from a first sub-block of the neural network;
• compressing, by the compression block, the input matrix into a compressed matrix to reduce utilization of hardware resources when transmitting and storing the compressed matrix, the compressed matrix having a reduced dimensionality relative to a dimensionality of the input matrix;
• outputting, by the compression block, the compressed matrix;
• retrieving, at a decompression block included in the neural network, the compressed matrix;
• decompressing, by the decompression block, the compressed matrix into a decompressed matrix, the decompressed matrix having a same dimensionality as the dimensionality of the input matrix;
• and
• outputting the decompressed matrix to a second sub-block of the neural network;
• wherein the compression block learns to optimize said compressing and the decompression block learns to optimize said decompressing based on feedback received from the neural network.

A non-transitory machine-readable medium storing a program executable by at least one processing unit of a computer system, the program comprising sets of instructions for:

• receiving, at a compression block included in a neural network, an input matrix from a first sub-block of the neural network;
• compressing, by the compression block, the input matrix into a compressed matrix to reduce utilization of hardware resources when transmitting and storing the compressed matrix, the compressed matrix having a reduced dimensionality relative to a dimensionality of the input matrix;
• outputting, by the compression block, the compressed matrix;
• retrieving, at a decompression block included in the neural network, the compressed matrix;
• decompressing, by the decompression block, the compressed matrix into a decompressed matrix, the decompressed matrix having a same dimensionality as the dimensionality of the input matrix;
• and
• outputting the decompressed matrix to a second sub-block of the neural network;
• wherein the compression block learns to optimize said compressing and the decompression block learns to optimize said decompressing based on feedback received from the neural network.