Method of selection of an action for an object using a neural network

A method of selection of an action for a object using a neural network, the method comprising:

• obtaining a plurality of predicted subsequent states of the object in the environment using an action model, a current state of the object in the environment and a plurality of actions, wherein the action model maps a plurality of states of the object in the environment and a plurality of actions performed by the object for each state to predicted subsequent states of the object in the environment;
• determining an action that maximizes a value of a target, wherein the target is based at least on a reward for each of the predicted subsequent states, wherein the target yi is γ·V(s′a)+R(s′a), wherein V(s′a) is a value of a predicted subsequent state based on a policy function V, γ is a discount factor, and R(s′a) is a determined reward R(s′a) for the predicted subsequent state;
• and
• performing the determined action.

A non-transitory machine readable medium having tangibly stored thereon executable instructions for execution by a processor of a control system for a object, wherein the executable instructions, when executed by the processor, cause the control system to:

• obtain a plurality of predicted subsequent states of the object in the environment using an action model, a current state of the object in the environment and a plurality of actions, wherein the action model maps a plurality of states of the object in the environment and a plurality of actions performed by the object for each state to predicted subsequent states of the object in the environment;
• determine an action that maximizes a value of a target, wherein the target is based at least on a reward for each of the predicted subsequent states, wherein the target yi is γ·V(s′a)+R(s′a), wherein V(s′a) is a value of a predicted subsequent state based on a policy function V, γ is a discount factor, and R(s′a) is a determined reward R(s′a) for the predicted subsequent state;
• and
• perform the determined action.

A method of training a first neural network for selection of an action for a object in an environment, the method comprising:

• predicting a plurality of subsequent states of the object in the environment from a plurality of states in a sample data set in accordance with an action model, wherein the action model maps a plurality of states of the object in the environment and a plurality of actions performed by the object for each state to predicted subsequent states of the object in the environment;
• determining a reward for each of the predicted subsequent states in the plurality of predicted subsequent states;
• determining an action that maximizes a value of a target, wherein the target is based at least on a reward for each of the predicted subsequent states, wherein the target yi is γ·V(s′a)+R(s′a), wherein V(s′a) is a value of a predicted subsequent state based on a policy function V, γ is a discount factor, and R(s′a) is a determined reward R(s′a) for the predicted subsequent state;
• calculating a training error as the difference between a value of the current state and the target;
• and
• updating parameters of the neural network to minimize the training error.