Systems and methods for lossless weight compression for back-to-back linear layers in transformers and other neural networks

Abstract

This specification discloses systems and methods for improving the inference and training efficiency of neural networks with back-to-back linear layers as present in transformer and diffusion models. Unlike standard weight compression and pruning techniques, the disclosed methods and systems use matrix inversion to reduce the total number of weights in an exact, mathematically equivalent way and thus without compromising model accuracy. The elimination of weights results in lower compute and memory complexities, and thus lower cost-per-token for generative AI. One embodiment of the invention reduces the total number of weights of a standard transformer model. Another embodiment of the invention discloses a simplified multi-head attention (MLA) scheme with lower memory and compute complexities.

Dependencies are inferred from the claim text.