Generating output examples using recurrent neural networks conditioned on bit values

A processing unit configured for generating an output example comprising a respective N-bit output value at each generation time step of a sequence of generation time steps, wherein the processing unit is configured to split the generating of the N-bit output value into two halves, to generate a first half of the N bits and then a second half of the N bits, wherein the processing unit is configured to, for each generation time step:

• process a first recurrent input comprising the N-bit output value at the preceding generation time step in the sequence using a recurrent neural network and in accordance with a hidden state of the recurrent neural network to generate a first score distribution over possible values for a first half of the N bits in the output value at the generation time step;
• select, using the first score distribution, values for the first half of the N bits of the output value;
• process a second recurrent input comprising (i) the N-bit output value at the preceding generation time step in the sequence and (ii) the values for the first half of the N bits using the recurrent neural network and in accordance with the same hidden state to generate a second score distribution over possible values for a second half of the N bits in the output value at the generation time step;
• and
• select, using the second score distribution, values for the second half of the N bits of the output value.

A system comprising one or more computers and one or more storage devices storing instructions that when executed by the one or more computers cause the one or more computers to perform operations for generating an output example comprising a respective N-bit output value at each generation time step of a sequence of generation time steps, the operations comprising, for each generation time step:

• processing a first recurrent input comprising the N-bit output value at the preceding generation time step in the sequence using a recurrent neural network and in accordance with a hidden state of the recurrent neural network to generate a first score distribution over possible values for a first half of the N bits in the output value at the generation time step;
• selecting, using the first score distribution, values for the first half of the N bits of the output value;
• processing a second recurrent input comprising (i) the N-bit output value at the preceding generation time step in the sequence and (ii) the values for the first half of the N bits using the recurrent neural network and in accordance with the same hidden state to generate a second score distribution over possible values for a second half of the N bits in the output value at the generation time step;
• and
• selecting, using the second score distribution, values for the second half of the N bits of the output value.