Automatic robot perception programming by imitation learning

A system comprising:

• means for identifying a first percept and a second percept from data gathered from a demonstration of a task;
• means for identifying a first number of planar surfaces of the first percept and a second number of planar surfaces of the second percept;
• means for calculating a first saliency value of the first percept based on the first number of planar surfaces and a second saliency value of the second percept based on the second number of planar surfaces, a higher number of planar surfaces corresponding to a higher saliency value, the first saliency value and the second saliency value representative of an information gain;
• means for mapping a trajectory based on the first percepts and the second percept, the first percept skewed based on the first saliency value, the second percept skewed based on the second saliency value;
• means for determining a plurality of variations of the trajectory and a collection of trajectories including the trajectory and the variations of the trajectory;
• means for imitating an action based on a first simulated signal from a first neural network of a first modality and a second simulated signal from a second neural network of a second modality;
• and
• means for executing a percept-action trajectory to perform the action.

A non-transitory computer readable medium comprising computer readable instructions that, when executed, cause one or more processors to at least:

• identify a first percept and a second percept from data gathered from a demonstration of a task;
• identify a first number of planar surfaces of the first percept and a second number of planar surfaces of the second percept;
• calculate a first unidimensional saliency value of the first percept based on the first number of planar surfaces and a second unidimensional saliency value of the second percept based on the second number of planar surfaces, a higher number of planar surfaces corresponding to a higher saliency value;
• map a trajectory based on the first percept and the second percept, the first percept skewed based on the first unidimensional saliency value, the second percept skewed based on the second unidimensional saliency value;
• determine a collection of trajectories including the trajectory and variations of the trajectory;
• imitate an action based on a first simulated signal from a first neural network of a first modality and a second simulated signal from a second neural network of a second modality, the action representative of a perceptual skill;
• and
• create source code for a robot to execute the perceptual skill.

A method comprising:

• identifying, by executing an instruction with a processor, a first percept and a second percept from data gathered from a demonstration of a task;
• identifying, by executing an instruction with the processor, a first number of planar surfaces of the first percept and a second number of planar surfaces of the second percept;
• calculating, by executing an instruction with the processor, a first unidimensional saliency value of the first percept based on the first number of planar surfaces and a second unidimensional saliency value of the second percept based on the second number of planar surfaces, a higher number of planar surfaces corresponding to a higher saliency value;
• mapping, by executing an instruction with the processor, a trajectory based on the first percept and the second percept, the first percept skewed based on the first unidimensional saliency value, the second percept skewed based on the second unidimensional saliency value;
• determining, by executing an instruction with the processor, a collection of trajectories including the trajectory and variations of the trajectory;
• imitating, by executing an instruction with the processor, an action based on a first simulated signal from a first neural network of a first modality and a second simulated signal from a second neural network of a second modality, the action representative of a perceptual skill;
• creating, by executing an instruction with the processor, source code for a robot to execute the perceptual skill;
• and
• transmitting the source code to the robot.