Binary Formats for Representing Persistent Parameters and Optimizer Parameters

Abstract

A method can include obtaining, by a computing system comprising one or more computing devices, a machine-learned model comprising a plurality of parameters having a binary format comprising X exponent bits and Y mantissa bits, wherein a precision of the binary format is not an integer power of two. The method can include, for each of a plurality of training iterations, obtaining, by the computing system, one or more input values for the machine-learned model. The method can include, for each of the plurality of training iterations, generating, by the computing system based at least in part on the input values and based at least in part on the parameters having the binary format, an output. The method can include, for each of the plurality of training iterations, updating, based at least in part on the output, at least one parameter of the plurality of parameters.

Dependencies are inferred from the claim text.