Method for controlling a robot and robot controller

A method for controlling a robot, comprising:

• providing demonstrations for performing each of a plurality of skills;
• training, from the demonstrations, a robot trajectory model for each skill of the skills, wherein each trajectory model is a hidden semi-Markov model having one or more initial states and one or more final states;
• training, from the demonstrations, (i) a precondition model for each skill including, for each initial state of the robot trajectory model of the skill, a probability distribution of robot configurations before executing the skill, and (ii) a final condition model for each skill including, for each final state of the robot trajectory model of the skill, a probability distribution of robot configurations after executing the skill;
• receiving a description of a task, wherein the task includes performing skills of the plurality of skills in sequence and/or branches;
• generating a composed robot trajectory model by:
• when two skills are to be performed in sequence in the task, cascading the robot trajectory models of the skills by:
• including the states of the trajectory models of the two skills in the composed robot trajectory model, and
• calculating a transition probability between each final state of the trajectory model of a first skill of the two skills and each initial state of the trajectory model of a second skill of the two skills as a function of a similarity between the probability distribution of the final condition model of the first skill for the final state of the first skill and the probability distribution of the initial model of the second skill for the initial state for the second skill,
• when two skills are performed in branches in the task, combining the robot trajectory models of the skills by:
• including the states of the trajectory models of the two skills in the composed robot trajectory model, and
• setting a transition probability between states of a first skill of
• the two skills and states of a second skill of the two skills to zero;
• and
• controlling the robot in accordance with the composed robot trajectory model to execute the task.

A robot controller configured to control a robot, the robot controller configured to:

• provide demonstrations for performing each of a plurality of skills;
• train, from the demonstrations, a robot trajectory model for each skill of the skills, wherein each trajectory model is a hidden semi-Markov model having one or more initial states and one or more final states;
• train, from the demonstrations, (i) a precondition model for each skill including, for each initial state of the robot trajectory model of the skill, a probability distribution of robot configurations before executing the skill, and (ii) a final condition model for each skill including, for each final state of the robot trajectory model of the skill, a probability distribution of robot configurations after executing the skill;
• receive a description of a task, wherein the task includes performing skills of the plurality of skills in sequence and/or branches;
• generate a composed robot trajectory model by:
• when two skills are to be performed in sequence in the task, cascading the robot trajectory models of the skills by:
• including the states of the trajectory models of the two skills in the composed robot trajectory model, and
• calculating a transition probability between each final state of the trajectory model of a first skill of the two skills and each initial state of the trajectory model of a second skill of the two skills as a function of a similarity between the probability distribution of the final condition model of the first skill for the final state of the first skill and the probability distribution of the initial model of the second skill for the initial state for the second skill,
• when two skills are performed in branches in the task, combining the robot trajectory models of the skills by:
• including the states of the trajectory models of the two skills in the composed robot trajectory model, and
• setting a transition probability between states of a first skill of the two skills and states of a second skill of the two skills to zero;
• and
• control the robot in accordance with the composed robot trajectory model to execute the task.

A non-transitory computer readable medium on which are stored instructions for controlling a robot, the instructions, when executed by a processor, causing the processor to perform the following:

• providing demonstrations for performing each of a plurality of skills;
• training, from the demonstrations, a robot trajectory model for each skill of the skills, wherein each trajectory model is a hidden semi-Markov model having one or more initial states and one or more final states;
• training, from the demonstrations, (i) a precondition model for each skill including, for each initial state of the robot trajectory model of the skill, a probability distribution of robot configurations before executing the skill, and (ii) a final condition model for each skill including, for each final state of the robot trajectory model of the skill, a probability distribution of robot configurations after executing the skill;
• receiving a description of a task, wherein the task includes performing skills of the plurality of skills in sequence and/or branches;
• generating a composed robot trajectory model by:
• when two skills are to be performed in sequence in the task, cascading the robot trajectory models of the skills by:
• including the states of the trajectory models of the two skills in the composed robot trajectory model, and
• calculating a transition probability between each final state of the trajectory model of a first skill of the two skills and each initial state of the trajectory model of a second skill of the two skills as a function of a similarity between the probability distribution of the final condition model of the first skill for the final state of the first skill and the probability distribution of the initial model of the second skill for the initial state for the second skill,
• when two skills are performed in branches in the task, combining the robot trajectory models of the skills by:
• including the states of the trajectory models of the two skills in the composed robot trajectory model, and
• setting a transition probability between states of a first skill of the two skills and states of a second skill of the two skills to zero;
• and
• controlling the robot in accordance with the composed robot trajectory model to execute the task.