Method and device for a computerized mechanical device

A system for training a computerized mechanical device's neural network dataset, comprising:

• at least one hardware processor, executing at least one neural network comprising a plurality of convolutional layers;
• at least one sensor electrically connected to an input of said at least one hardware processor;
• and
• at least one controller, connected to an output of said at least one hardware processor;
• wherein said at least one hardware processor is adapted to:
• receive data documenting a plurality of actions demonstrated by a demonstrating actuator performing a target task in a plurality of initial iterations;
• calculate using said data a neural network dataset having a plurality of neural network parameters and used for mimicking said demonstrated plurality of actions in performing said target task;
• gather, in a plurality of reward training iterations of a robotic actuator performing said target task according to said neural network dataset, a plurality of scores given by an instructor to a plurality of world states, each world state comprising at least one sensor output value received from said at least one sensor and associated with said performance of said robotic actuator;
• calculate, using said plurality of scores, a reward neural network dataset having a second plurality of neural network parameters;
• compute, through machine learning, a reward function from said reward neural network dataset;
• receive in each of a plurality of policy training iterations a reward value computed by applying said reward function to another world state, while said robotic actuator performs said target task according to said neural network dataset, wherein said another world state comprising at least one sensor output value received from said at least one sensor;
• gather in a plurality of safety training iterations a plurality of safety scores given by a safety instructor to a plurality of safety states, each safety state comprising at least one other sensor output value, while said robotic actuator performs said target task according to said neural network dataset;
• and
• calculate using said plurality of safety scores a safety neural network dataset having a third plurality of neural network parameters and used for computing a safety function;
• update at least some of said plurality of neural network parameters based on said received reward value of each of said plurality of policy training iteration;
• and
• output said updated neural network dataset;
• wherein said at least some of said plurality of neural network parameters are updated by:
• receiving in each of said plurality of policy training iterations a safety value computed by applying said safety function to said another world state, while said robotic actuator performs said target task according to said neural network dataset;
• and
• identifying at least one safe controller action subject to said safety value being less than an identified threshold safety value.

A method for a computerized mechanical device, comprising:

• accessing a neural network data set generated by:
• receiving data documenting a plurality of actions demonstrated by a demonstrating actuator performing a target task in a plurality of initial iterations;
• calculating using said data a neural network dataset having a plurality of neural network parameters and used for mimicking said demonstrated plurality of actions in performing said target task;
• gathering, in a plurality of reward training iterations of a robotic actuator performing said target task according to said neural network dataset, a plurality of scores given by an instructor to a plurality of world states, each world state comprising at least one sensor output value associated with said performance of said robotic actuator;
• calculating, using said plurality of scores, a reward neural network dataset having a second plurality of neural network parameters;
• computing, through machine learning, a reward function from said reward neural network dataset;
• receiving in each of a plurality of policy training iterations a reward value computed by applying said reward function to another world state while said robotic actuator performs said target task according to said neural network dataset, wherein said another world state comprising at least one sensor output value;
• gathering in a plurality of safety training iterations a plurality of safety scores given by a safety instructor to a plurality of safety states, each safety state comprising at least one other sensor output value, while said robotic actuator performs said target task according to said neural network dataset;
• and
• calculating using said plurality of safety scores a safety neural network dataset having a third plurality of neural network parameters and used for computing a safety function;
• updating at least some of said plurality of neural network parameters based on said received reward value of each of said plurality of policy training iteration;
• and
• outputting said updated neural network dataset;
• receiving a plurality of sensor output values;
• and
• instructing at least one controller to perform one or more of an identified set of controller actions according to said updated neural network dataset in response to receiving said plurality of sensor output values;
• wherein updating said at least some of said plurality of neural network parameters further comprises:
• receiving in each of said plurality of policy training iterations a safety value computed by applying said safety function to said another world state, while said robotic actuator performs said target task according to said neural network dataset;
• and
• identifying at least one safe controller action subject to said safety value being less than an identified threshold safety value.