Back to results
US 2026/0236773 A1Published application
Controlling agents using proto-goal pruning
AssigneeDeepMind Technologies Limited
InventorsTom Schaul, Akhil Bagaria
Life of this patent
Estimating the term…
Abstract
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for controlling agents using proto-goal pruning.
Claims28
A method for controlling an agent interacting with an environment to perform a task, the method comprising, at each of a plurality of iterations:
- identifying, from a plurality of proto-goals that were achieved in transitions stored in a replay memory, a plurality of candidate proto-goals for the agent;
- selecting, as plausible goals for the iteration, a plurality of the candidate proto-goals;
- selecting a goal from the plurality of plausible goals;
- and
- controlling the agent using a policy neural network conditioned on the selected goal to generate a new trajectory that comprises a sequence of transitions, data identifying the selected goal, and a goal reward indicating whether the selected goal was achieved in the new trajectory, each transition comprising:
- (i) an observation characterizing a state of the environment,
- (ii) data identifying each proto-goal that was achieved when the environment was in the state, and
- (iii) an action performed by the agent in response to the observation.
5 more elements
A system comprising:
- one or more computers;
- and
- one or more storage devices communicatively coupled to the one or more computers, wherein the one or more storage devices store instructions that, when executed by the one or more computers, cause the one or more computers to perform operations for controlling an agent interacting with an environment to perform a task, the operations comprising, at each of a plurality of iterations;
- identifying, from a plurality of proto-goals that were achieved in transitions stored in a replay memory, a plurality of candidate proto-goals for the agent;
- selecting, as plausible goals for the iteration, a plurality of the candidate proto-goals;
- selecting a goal from the plurality of plausible goals;
- and
- controlling the agent using a policy neural network conditioned on the selected goal to generate a new trajectory that comprises a sequence of transitions, data identifying the selected goal, and a goal reward indicating whether the selected goal was achieved in the new trajectory, each transition comprising:
- (i) an observation characterizing a state of the environment,
- (ii) data identifying each proto-goal that was achieved when the environment was in the state, and
- (iii) an action performed by the agent in response to the observation.
8 more elements
One or more non-transitory computer storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations for controlling an agent interacting with an environment to perform a task, the operations comprising, at each of a plurality of iterations:
- identifying, from a plurality of proto-goals that were achieved in transitions stored in a replay memory, a plurality of candidate proto-goals for the agent;
- selecting, as plausible goals for the iteration, a plurality of the candidate proto-goals;
- selecting a goal from the plurality of plausible goals;
- and
- controlling the agent using a policy neural network conditioned on the selected goal to generate a new trajectory that comprises a sequence of transitions, data identifying the selected goal, and a goal reward indicating whether the selected goal was achieved in the new trajectory, each transition comprising:
- (i) an observation characterizing a state of the environment,
- (ii) data identifying each proto-goal that was achieved when the environment was in the state, and
- (iii) an action performed by the agent in response to the observation.
5 more elements
Dependencies are inferred from the claim text.