Loading patent…
AssigneeGoogle Inc.
InventorsDeniz Oktay, Saurabh Singh, Johannes Balle, Abhinav Shrivistava
Estimating the term…
Example aspects of the present disclosure are directed to systems and methods that learn a compressed representation of a machine-learned model (e.g., neural network) via representation of the model parameters within a reparameterization space during training of the model. In particular, the present disclosure describes an end-to-end model weight compression approach that employs a latent-variable data compression method. The model parameters (e.g., weights and biases) are represented in a “latent” or “reparameterization” space, amounting to a reparameterization. In some implementations, this space can be equipped with a learned probability model, which is used first to impose an entropy penalty on the parameter representation during training, and second to compress the representation using arithmetic coding after training. The proposed approach can thus maximize accuracy and model compressibility jointly, in an end-to-end fashion, with the rate-error trade-off specified by a hyperparameter.
A computer-implemented method to learn a compressed representation of a machine-learned model via a representation within a reparameterization space, the method comprising:
One or more non-transitory computer-readable media that store instructions that, when executed by one or more computing devices, cause the one or more computing devices to perform operations for learning a compressed representation of a machine-learned model via a representation within a reparameterization space, the operations comprising:
A computing device comprising:
Dependencies are inferred from the claim text.