Method and device for training manipulation skills of a robot system

A robot system trainable via physical input from a human trainer, regarding a set of skills to be learned by the robot system for manipulation of an object, wherein each skill is learned as a skill model, the robot system configured to:

• for each skill of the set of skills:
• receive physical input from a human trainer regarding the skill to be learned by the robot, wherein the physical input includes a set of kinesthetic demonstrations on the robot system;
• determine for the skill model of the skill a set of task parameters including determining for each of the task parameters of the set of task parameters if the task parameter is an attached task parameter, which is related to an object being part of the kinesthetic demonstration, or if the task parameter is a free task parameter, which is not related to a physical object, wherein in the determining, when the skill is a transition skill, at least one of the set of task parameters for the skill model is determined to be a free task parameter which is explicitly set by a user and directly influences an outcome of the transition skill;
• obtain data for each of the task parameters of the set of task parameters from the set of kinesthetic demonstrations;
• and
• train the skill model with the set of task parameters and the data obtained for each of the task parameters;
• wherein the set of skills includes at least one core skill and at least one transition skill, each core skill includes a desired final state of the object and each transition skill is a skill where a final state of the object is freely chosen during the demonstrations;
• and
• wherein the robot system is further configured to:
• determine a sequence of the skills, the sequence including the at least one core skill and the at least one transition skill, the sequence being for the manipulation of the object, and
• generate a reference trajectory based on the sequence;
• and
• wherein the robot system is further configured to:
• learn a precondition model of each skill, which refers to a relative relations between robot system and end-effector of the robot system, and an object at an initial time, wherein the precondition model is defined by:
• γ1,α(s,pF)≙{(μ1,o(p),Σ1,o(p)),∀p∈TPα\{o}}o∈OαUFα
• where (μ1,o(p), Σ1,o(p)) is a Gaussian distribution at initial time 1;
• wherein α is the skill, Oa are task parameters for objects that are relevant for the skill, Fa are the free parameters for the skill, TPa are all task parameters, s is a current state of the robot system, and pF is a value for the free task parameters, o is either a task parameter for the objects that are relevant for the skill or a free parameter for the skill, and p is one of the task parameters excluding o.

A computer for controlling a robot system of manipulation of an object, the robot system being trained to perform a set of skills for the manipulation of the object, wherein each of the skills is learned as a skill model, the skill model being a TP-HSMM, Task Parameterized Hidden semi-Markov Model, combined with a TP-GMM, Task Parameterized Gaussian Mixture Model, the computer configured to:

• for each skill of the set of skills:
• receive physical input from a human trainer regarding the skill to be learned by the robot, wherein the physical input includes a set of kinesthetic demonstrations on the robot system;
• determine for the skill model a set of task parameters including determining for each of the task parameters of the set of task parameters if the task parameter is an attached task parameter, which is related to an object being part of the kinesthetic demonstration, or if the task parameter is a free task parameter, which is not related to a physical object, wherein in the determining, when the skill is a transition skill, at least one of the set of task parameters for the skill model is determined to be a free task parameter which is explicitly set by a user and directly influences an outcome of the transition skill;
• obtain data for each of the task parameters of the set of task parameters from the set of kinesthetic demonstrations;
• and
• train the skill model with the set of task parameters and the data obtained for each of the task parameters;
• wherein the set of skills includes at least one core skill and at least one transition skill, each core skill includes a desired final state of the object and each transition skill is a skill where a final state of the object is freely chosen during the demonstrations;
• and
• wherein the computer is further configured to:
• determine a sequence of the skills, the sequence including the at least one core skill and the at least one transition skill, the sequence being for the manipulation of the object, and
• generate a reference trajectory based on the sequence;
• and
• wherein the computer is further configured to:
• learn a precondition model of each skill, which refers to a relative relations between robot system and end-effector of the robot system, and an object at an initial time, wherein the precondition model is defined by:
• γ1,α(s,pF)≙{(μ1,o(p),Σ1,o(p)),∀p∈TPα\{o}}o∈OαUFα
• where (μ1,o(p), Σ1,o(p)) is a Gaussian distribution at initial time 1;
• wherein α is the skill, Oa are task parameters for objects that are relevant for the skill, Fa are the free parameters for the skill, TPa are all task parameters, s is a current state of the robot system, and pF is a value for the free task parameters, o is either a task parameter for the objects that are relevant for the skill or a free parameter for the skill, and p is one of the task parameters excluding o.

A non-transitory machine-readable storage medium on which is stored a computer program for training a robot system for manipulation of an object, the robot system being able to perform a set of skills for the manipulation of the object, wherein each of the skills is learned as a skill model, the skill model being a TP-HSMM, Task Parameterized Hidden semi-Markov Model, combined with a TP-GMM, Task Parameterized Gaussian Mixture Model, the computer program, when executed by a computer, causing the computer to perform the following steps:

• for each skill of the set of skills:
• receiving physical input from a human trainer regarding a skill to be learned by the robot, wherein the physical input includes a set of kinesthetic demonstrations on the robot system;
• determining for the skill model of the skill a set of task parameters including determining for each of the task parameters of the set of task parameters if the task parameter is an attached task parameter, which is related to an object being part of the kinesthetic demonstration, or if the task parameter is a free task parameter, which is not related to a physical object, wherein in the determining, when the skill is a transition skill, at least one of the set of task parameters for the skill model is determined to be a free task parameter which is explicitly set by a user and directly influences an outcome of the transition skill;
• obtaining data for each of the task parameters of the set of task parameters from the set of kinesthetic demonstrations;
• and
• training the skill model with the set of task parameters and the data obtained for each of the task parameters;
• wherein the set of skills includes at least one core skill and at least one transition skill, each core skill includes a desired final state of the object and each transition skill is a skill where a final state of the object is freely chosen during the demonstrations;
• and
• wherein the method further includes:
• determining a sequence of the skills, the sequence including the at least one core skill and the at least one transition skill, the sequence being for the manipulation of the object, and
• generating a reference trajectory based on the sequence;
• and
• wherein the method further comprises:
• learn a precondition model of each skill, which refers to a relative relations between robot system and end-effector of the robot system, and an object at an initial time, wherein the precondition model is defined by:
• γ1,α(s,pF)≙{(μ1,o(p),Σ1,o(p)),∀p∈TPα\{o}}o∈OαUFα
• where (μ1,o(p), Σ1,o(p)) is a Gaussian distribution at initial time 1;
• wherein α is the skill, Oa are task parameters for objects that are relevant for the skill, Fa are the free parameters for the skill, TPa are all task parameters, s is a current state of the robot system, and pF is a value for the free task parameters, o is either a task parameter for the objects that are relevant for the skill or a free parameter for the skill, and p is one of the task parameters excluding o.

A method of using a robot system for manipulation of objects, the method comprising the following steps:

• providing the robot system;
• training the robot system for manipulation of an object, the robot system being trained to perform a set of skills for the manipulation of the object, wherein each of the skills is learned as a skill model, the skill model being a TP-HSMM, Task Parameterized Hidden semi-Markov Model, combined with a TP-GMM, Task Parameterized Gaussian Mixture Model, the robot system being trained to perform the set of skills by:
• for each skill of the set of skills
• receiving physical input from a human trainer regarding the skill to be learned by the robot, wherein the physical input includes a set of kinesthetic demonstrations on the robot system;
• determining for the skill model for the skill a set of task parameters including determining for each of the task parameters of the set of task parameters if the task parameter is an attached task parameter, which is related to an object being part of the kinesthetic demonstration, or if the task parameter is a free task parameter, which is not related to a physical object, wherein in the determining, when the skill is a transition skill, at least one of the set of task parameters for the skill model is determined to be a free task parameter which is explicitly set by a user and directly influences an outcome of the transition skill;
• obtaining data for each of the task parameters of the set of task parameters from the set of kinesthetic demonstrations;
• and
• training the skill model with the set of task parameters and the data obtained for each of the task parameters;
• wherein the set of skills includes at least one core skill and at least one transition skill, each core skill includes a desired final state of the object and each transition skill is a skill where a final state of the object is freely chosen during the demonstrations;
• and
• determining a sequence of the skills, the sequence including the at least one core skill and the at least one transition skill, the sequence being for the manipulation of the object;
• and
• performing, by the robot system, the determined sequence including the at least one core skill and the at least one transition skill, the sequence being for the manipulation of the object;
• and
• wherein the method further comprises:
• learning a precondition model of each skill, which refers to a relative relations between robot system and end-effector of the robot system, and an object at an initial time, wherein the precondition model is defined by:
• γ1,α(s,pF)≙{(μ1,o(p),Σ1,o(p)),∀p∈TPα\{o}}o∈OαUFα
• where (μ1,o(p), Σ1,o(p)) is a Gaussian distribution at initial time 1;
• wherein α is the skill, Oa are task parameters for objects that are relevant for the skill, Fa are the free parameters for the skill, TPa are all task parameters, s is a current state of the robot system, and pF is a value for the free task parameters, o is either a task parameter for the objects that are relevant for the skill or a free parameter for the skill, and p is one of the task parameters excluding o.