# US 10,872,299 B2 · Memory augmented generative temporal models > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,872,299 B2 - Document ID: US10872299B2 - Status: Granted - Kind: B2 - Application number: 16/459,113 - Filed: 2019-07-01 - Published: 2019-10-24 - Granted: 2020-12-22 - Earliest claimed: 2017-02-06 - Assignees: DeepMind Technologies Limited - Applicants: DeepMind Technologies Limited - Inventors: Gregory Duncan Wayne; Chia-Chun Hung; Mevlana Celaleddin Gemici; Adam Anthony Santoro - Web page: https://patentgrep.com/patent/US10872299B2 ## CPC - G06N 3/047: Probabilistic or stochastic networks - G06N 3/0475: Generative networks - G06N 3/044: Recurrent networks, e.g. Hopfield networks - G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] - G06N 3/045: Combinations of networks - G06N 3/0455: Auto-encoder networks; Encoder-decoder networks - G06N 3/049: Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs - G06N 3/06: Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons - G06N 3/09: Supervised learning - G06N 20/00: Machine learning - G05B 2219/33025: Recurrent artificial neural network - G06F 16/908: using metadata automatically derived from the content - G06N 3/084: Backpropagation, e.g. using gradient descent - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating sequences of predicted observations, for example images. In one aspect, a system comprises a controller recurrent neural network, and a decoder neural network to process a set of latent variables to generate an observation. An external memory and a memory interface subsystem is configured to, for each of a plurality of time steps, receive an updated hidden state from the controller, generate a memory context vector by reading data from the external memory using the updated hidden state, determine a set of latent variables from the memory context vector, generate a predicted observation by providing the set of latent variables to the decoder neural network, write data to the external memory using the latent variables, the updated hidden state, or both, and generate a controller input for a subsequent time step from the latent variables. ## Claims ### Claim 1 A system comprising one or more computers and one or more storage devices storing instructions that when executed by the one or more computers cause the one or computers to implement: a controller recurrent neural network comprising controller recurrent neural network instructions that when executed by the one or more computers cause the one or more computers to perform operations comprising, for each of a plurality of time steps: receiving a controller neural network input for the time step, wherein the controller neural network input for the time step comprises a set of latent variables generated by a memory interface subsystem at a previous time step that collectively define a compressed representation of a predicted observation characterizing an environment at the previous time step; and processing the controller neural network input for the time step in accordance with a current hidden state to generate an updated hidden state for the time step; a decoder neural network comprising decoder neural network instructions that when executed by the one or more computers cause the one or more computers to perform operations comprising: receiving a decoder neural network input comprising a set of latent variables that collectively define a compressed representation of a predicted observation characterizing the environment; and processing the decoder neural network input comprising the set of latent variables to generate the predicted observation characterizing the environment; an external memory; and the memory interface subsystem comprising memory interface subsystem instructions that when executed by the one or more computers cause the one or more computers to perform operations comprising, for each of the time steps: receiving the updated hidden state for the time step; generating a memory context vector by reading data from the external memory using the updated hidden state; determining, from at least the memory context vector, a set of latent variables for the time step that collectively define a compressed representation of a predicted observation characterizing the environment at the time step; generating the predicted observation characterizing the environment at the time step by providing the set of latent variables as input to the decoder neural network; writing data to the external memory using the set of latent variables, the updated hidden state, or both; and generating a controller neural network input for a subsequent time step from the set of latent variables for the time step, wherein the controller neural network input for the subsequent time step comprises the set of latent variables for the time step. ### Claim 2 (depends on 1) The system of claim 1, wherein writing data to the external memory comprises: writing the latent variables for the time step to the external memory. ### Claim 3 (depends on 2) The system of claim 2, wherein writing data to the external memory comprises: writing the latent variables to a specified location in the external memory. ### Claim 4 (depends on 1) The system of claim 1, wherein writing data to the external memory comprises: generating a set of writing weights, wherein the set of writing weights includes a respective writing weight for each of a plurality of locations in the external memory; and writing data defined by the updated hidden state to the locations in the external memory in accordance with the writing weights. ### Claim 5 (depends on 1) The system of claim 1, wherein generating a memory context vector by reading data from the external memory using the updated hidden state comprises, for each of one or more read heads: generating a set of reading weights for the read head from the updated hidden state, wherein the set of reading weights includes a respective reading weight for each of a plurality of locations in the external memory; and reading data from the plurality of locations in the external memory in accordance with the reading weights. ### Claim 6 (depends on 5) The system of claim 5, wherein the reading weights are position-based addressing weights. ### Claim 7 (depends on 5) The system of claim 5, wherein the reading weights are content-based addressing weights. ### Claim 8 (depends on 5) The system of claim 5, wherein the reading weights are a combination of position-based addressing weights and content-based addressing weights. ### Claim 9 (depends on 1) The system of claim 1, wherein determining a set of latent variables for the time step comprises: mapping, using a prior map, a prior map input comprising the memory context vector to, for each of the latent variables, parameters of a prior distribution over possible latent variable values for the latent variable; and sampling the latent variables from the prior distributions for the latent variables in accordance with the distribution parameters. ### Claim 10 (depends on 9) The system of claim 9, wherein the prior map input further comprises a combination of a set of latent variables for a preceding time step and the updated hidden state. ### Claim 11 (depends on 10) The system of claim 10, wherein the prior map is one or more linear neural network layers that are configured to map the prior map input to the distribution parameters. ### Claim 12 (depends on 1) The system of claim 1, wherein determining a set of latent variables for the time step comprises: receiving an actual observation for the time step; processing the memory context vector and the actual observation using a posterior map neural network comprising posterior map neural network instructions that when executed by the one or more computers cause the one or more computers to perform operations comprising: generating, for each of the latent variables, parameters of a posterior distribution over possible latent variable values for the latent variable; and sampling the latent variables from the posterior distributions for the latent variables in accordance with the distribution parameters. ### Claim 13 (depends on 1) The system of claim 1, wherein generating the controller neural network input for the subsequent time step from the latent variables for the time step comprises: combining the latent variables for the time step and an external context vector to generate the controller neural network input for the subsequent time step. ### Claim 14 (depends on 1) The system of claim 1, wherein generating the controller neural network input for the subsequent time step from the latent variables for the time step comprises: combining the latent variables for the time step and data from the external memory to generate the controller neural network input for the subsequent time step. ### Claim 15 (depends on 1) The system of claim 1, wherein generating the controller neural network input for the subsequent time step from the latent variables for the time step comprises: combining the latent variables for the time step, data read from the external memory, and an external context vector to generate the controller neural network input for the subsequent time step. ### Claim 16 (depends on 1) The system of claim 1, wherein the controller neural network input for the time step comprises latent variables for a preceding time step and an external context vector for the time step. ### Claim 17 (depends on 16) The system of claim 16, wherein the external context vector represents one or more external variables that influence characteristics of observations generated by the system. ### Claim 18 A method performed by one or more data processing apparatus, the method comprising, for each of a plurality of time steps: receiving a controller neural network input for the time step, wherein the controller neural network input for the time step comprises a set of latent variables generated at a previous time step that collectively define a compressed representation of a predicted observation characterizing an environment at the previous time step; processing the controller neural network input for the time step using a controller recurrent neural network, in accordance with a current hidden state of the controller recurrent neural network, to generate an updated hidden state of the controller recurrent neural network for the time step; generating a memory context vector by reading data from an external memory using the updated hidden state of the controller recurrent neural network for the time step; determining, from at least the memory context vector, a set of latent variables for the time step that collectively define a compressed representation of a predicted observation characterizing the environment at the time step; generating the predicted observation characterizing the environment at the time step, comprising: receiving a decoder neural network input comprising the set of latent variables; and processing the decoder neural network input comprising the set of latent variables to generate the predicted observation characterizing the environment at the time step; writing data to the external memory using the set of latent variables, the updated hidden state, or both; and generating a controller neural network input for a subsequent time step from the set of latent variables for the time step, wherein the controller neural network input for the subsequent time step comprises the set of latent variables for the time step. ### Claim 19 One or more non-transitory computer storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations comprising, for each of a plurality of time steps: receiving a controller neural network input for the time step, wherein the controller neural network input for the time step comprises a set of latent variables generated at a previous time step that collectively define a compressed representation of a predicted observation characterizing an environment at the previous time step; processing the controller neural network input for the time step using a controller recurrent neural network, in accordance with a current hidden state of the controller recurrent neural network, to generate an updated hidden state of the controller recurrent neural network for the time step; generating a memory context vector by reading data from an external memory using the updated hidden state of the controller recurrent neural network for the time step; determining, from at least the memory context vector, a set of latent variables for the time step that collectively define a compressed representation of a predicted observation characterizing the environment at the time step; generating the predicted observation characterizing the environment at the time step, comprising: receiving a decoder neural network input comprising the set of latent variables; and processing the decoder neural network input comprising the set of latent variables to generate the predicted observation characterizing the environment at the time step; writing data to the external memory using the set of latent variables, the updated hidden state, or both; and generating a controller neural network input for a subsequent time step from the set of latent variables for the time step, wherein the controller neural network input for the subsequent time step comprises the set of latent variables for the time step.