US 2017/0323201 A1 · AUGMENTING NEURAL NETWORKS WITH EXTERNAL MEMORY USING REINFORCEMENT LEARNING

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for augmenting neural networks with an external memory using reinforcement learning. One of the methods includes providing an output derived from the system output portion of the neural network output as a system output in the sequence of system outputs; selecting a memory access process from a predetermined set of memory access processes for accessing the external memory from the reinforcement learning portion of the neural network output; writing and reading data from locations in the external memory in accordance with the selected memory access process using the differentiable portion of the neural network output; and combining the data read from the external memory with a next system input in the sequence of system inputs to generate a next neural network input in the sequence of neural network inputs.

Claims20

Dependent claims

An augmented neural network system for processing a sequence of system inputs to generate a sequence of system outputs, the augmented neural network system comprising:

a neural network, wherein the neural network is configured to receive a sequence of neural network inputs and to process each neural network input to generate a neural network output from the neural network input, the neural network output comprising:
a system output portion,
a reinforcement learning portion, and
a differentiable portion;
an external memory;
and
a memory interface subsystem, wherein the memory interface subsystem is configured to perform operations comprising, for each of the neural network outputs:
providing an output derived from the system output portion of the neural network output as a system output in the sequence of system outputs;
selecting a memory access process from a predetermined set of memory access processes for accessing the external memory from the reinforcement learning portion of the neural network output;
writing and reading data from locations in the external memory in accordance with the selected memory access process using the differentiable portion of the neural network output;
and
combining the data read from the external memory with a next system input in the sequence of system inputs to generate a next neural network input in the sequence of neural network inputs.

9 more elements

A method of processing a sequence of system inputs to generate a sequence of system outputs, the method comprising:

providing a neural network input to a neural network, wherein the neural network is configured to receive the neural network input and to process the neural network input to generate a neural network output from the neural network input, the neural network output comprising:
a system output portion,
a reinforcement learning portion, and
a differentiable portion;
providing an output derived from the system output portion of the neural network output as a system output in the sequence of system outputs;
selecting a memory access process from a predetermined set of memory access processes for accessing an external memory from the reinforcement learning portion of the neural network output;
writing and reading data from locations in the external memory in accordance with the selected memory access process using the differentiable portion of the neural network output;
and
combining the data read from the external memory with a next system input in the sequence of system inputs to generate a next neural network input.

6 more elements

Dependencies are inferred from the claim text.

Augmenting neural networks with external memory using reinforcement learning

Life of this patent

Augmenting neural networks with external memory using reinforcement learning

Life of this patent

Abstract

Claims20