Data compression by local entropy encoding

A system for decompressing data, the system comprising:

• a decompression subsystem configured to perform operations comprising:
• receiving a compressed representation of data comprising a compressed representation of each of one or more code symbol subsets of an ordered collection of code symbols representing the data, wherein the compressed representation of a code symbol subset comprises:
• (i) an entropy encoded representation of the code symbol subset, and (ii) data indicating a code symbol probability distribution used to entropy encode the code symbol subset;
• determining each code symbol subset by decoding the entropy encoded representation of the code symbol subset based on the code symbol probability distribution used to entropy encode the code symbol subset;
• determining a representation of the data as an ordered collection of code symbols based on the code symbol subsets;
• and
• a decoder neural network configured to perform operations comprising:
• receiving the representation of the data as an ordered collection of code symbols determined by the decompression subsystem;
• processing the ordered collection of code symbols in accordance with current values of a set of decoder neural network parameters to generate an output defining a reconstruction of the data.

A computer-implemented method for compressing data, the method comprising:

• receiving data to be compressed;
• processing the data by an encoder neural network and in accordance with current values of a set of encoder neural network parameters to generate an output including a representation of the data as an ordered collection of code symbols;
• identifying a partition of the collection of code symbols into one or more code symbol subsets;
• for each code symbol subset:
• identifying, from a dictionary of multiple code symbol probability distributions, and based on the code symbol subset, a code symbol probability distribution for the code symbol subset;
• generating an entropy encoded representation of the code symbol subset by entropy encoding the code symbol subset using the code symbol probability distribution identified for the code symbol subset;
• determining a compressed representation of the code symbol subset, the compressed representation comprising:
• (i) the entropy encoded representation of the code symbol subset, and (ii) data indicating a code symbol probability distribution used to entropy encode the code symbol subset;
• determining a compressed representation of the data based on the compressed representations of the code symbol subsets.

A non-transitory computer readable storage medium storing instructions executable by a data processing apparatus and that upon such execution causes the data processing apparatus to perform operations comprising:

• receiving data to be compressed;
• processing the data by an encoder neural network and in accordance with current values of a set of encoder neural network parameters to generate an output including a representation of the data as an ordered collection of code symbols;
• identifying a partition of the collection of code symbols into one or more code symbol subsets;
• for each code symbol subset:
• identifying, from a dictionary of multiple code symbol probability distributions, and based on the code symbol subset, a code symbol probability distribution for the code symbol subset;
• generating an entropy encoded representation of the code symbol subset by entropy encoding the code symbol subset using the code symbol probability distribution identified for the code symbol subset;
• determining a compressed representation of the code symbol subset, the compressed representation comprising:
• (i) the entropy encoded representation of the code symbol subset, and (ii) data indicating a code symbol probability distribution used to entropy encode the code symbol subset;
• determining a compressed representation of the data based on the compressed representations of the code symbol subsets.