Apparatus and methods for control of robot actions based on corrective user inputs

A system, comprising:

• one or more data processors;
• and
• a non-transitory computer readable storage medium containing instructions which when executed on the one or more data processors, cause the processor to:
• identify a first context-variable value for a context variable detected by a robot at a first sensory-detection time;
• access a policy comprising one or more parameters configured to map the context variable to a robot action variable;
• determine that a first robot action characterized by a first value of the robot action variable was performed at an action time in response to detection of the first context-variable value, the first robot action being in accordance with application of the policy;
• determine that a user input was received at an input time configured to correspond to the action time, wherein user input data derived from the user input at least partly defines a corrective command that specifies a corrective robot action for physical performance, the user input being indicative of at least partial dissatisfaction with the first robot action, wherein the corrective command defined by the user input data is configured to minimize an error associated with the robot action;
• modify the policy based on the corrective command and the st context-variable value;
• and
• cause the modified policy to be used to:
• determine a second robot action characterized by a second value of the robot action variable based on a second context-variable value for the context variable detected at a second sensory-detection time;
• and
• initiate performance of the second robot action in accordance with the second value of the robot action variable.

A computer-program product tangibly embodied in a non-transitory machine-readable storage medium, including instructions configured to cause one or more data processors to:

• identify a first context-variable value for a context variable detected by a robot at a first sensory-detection time;
• access a policy comprising one or more parameters configured to map the context variable to a robot action variable;
• determine that a first robot action characterized by a first value of the robot action variable was performed at an action time in response to detection of the first context-variable value, the first robot action being in accordance with application of the policy;
• determine that a user input was received at an input time corresponding to the action time, wherein user input data derived from the user input at least partly defines a corrective command that specifies a corrective robot action for physical performance, the user input being indicative of at least partial dissatisfaction with the first robot action;
• modify the policy based on the corrective command and the first context-variable value;
• and
• cause the modified policy to be used to:
• determine a second robot action characterized by a second value of the robot action variable based on a second context-variable value for the context variable detected at a second sensory-detection time;
• and
• initiate performance of the second robot action in accordance with the second value of the robot action variable;
• wherein the second value of the robot action variable comprises a combination of the first robot action and the corrective action defined by the user input data, the combination being configured to result in a desired robot action.