Machine learning device, robot system, and machine learning method for learning motion of robot engaged in task performed by human and robot in cooperate with each other
AssigneeFanuc Corporation
InventorsShunichi Ozaki, Hiroji Nishi
Life of this patent
Estimating the term…
Abstract
A machine learning device for learning a motion of a robot engaged in a task performed by a human and a robot in cooperation with each other, including a state observation unit that observes a state variable indicating a state of the robot when the human and the robot cooperate with each other and perform a task; a reward calculation unit that calculates a reward based on control data and the state variable for controlling the robot and on an action of the human; and a value function update unit that updates an action value function for controlling a motion of the robot, based on the reward and the state variable.
Claims12
A robot system, comprising:
- a robot configured to perform a task in cooperation with a human;
- a robot control unit controlling a motion of the robot;
- a machine learning device for learning the motion of the robot engaged in the task performed by the human and the robot in cooperation with each other, the machine learning device comprising:
- a state observation unit configured to observe a state variable indicating a state of the robot when the human and the robot cooperate with each other and perform the task;
- a reward calculation unit configured to calculate a reward based
- (1) on control data and the state variable for controlling the robot, and
- (2) on a reaction of the human to the robot in response to the motion of the robot when the task is performed by the human and the robot cooperating with each other;
- and
- a value function update unit configured to update an action value function for controlling the motion of the robot, based on the reward and the state variable, wherein the machine learning device is configured to learn the motion of the robot by analyzing a distribution of feature points or workpieces after the human and the robot have cooperated with each other and performed the task;
- and
- a task intention recognition unit configured to receive outputs of at least one of a camera, a force sensor, a tactile sensor, a microphone, or an input device, and recognize an intention regarding the task,
- wherein the state variable inputted to the state observation unit of the machine learning device is an output of the task intention recognition unit,
- wherein the task intention recognition unit is configured to
- convert a positive reward based on the reaction of the human into a state variable set for the positive reward and output the state variable to the state observation unit, and
- convert a negative reward based on the reaction of the human into a state variable set for the negative reward and output the state variable to the state observation unit, and
- wherein the reaction of the human is received by at least one of the camera, the force sensor, the tactile sensor, the microphone, or the input device, during a time when the task is being performed by the human and the robot cooperating with each other.
A machine learning method for learning a motion of a robot engaged in a task performed by a human and the robot in cooperation with each other, the machine learning method comprising:
- observing a state variable indicating a state of the robot when the human and the robot cooperate with each other and perform a task;
- calculating a reward based
- (1) on control data and the state variable for controlling the robot, and
- (2) on a reaction of the human to the robot in response to the motion of the robot when the task is performed by the human and the robot cooperating with each other;
- and
- updating an action value function for controlling a motion of the robot, based on the reward and the state variable,
- the machine learning method further comprising:
- learning the motion of the robot by analyzing a distribution of feature points or workpieces after the human and the robot have cooperated with each other and performed the task;
- during a time when the task is being performed by the human and the robot cooperating with each other, receiving the reaction of the human by at least one or a camera, a force sensor, a tactile sensor, a microphone, or an input device, to recognize an intention regarding the task;
- converting a positive reward based on the reaction of the human into a state variable set for the positive reward and outputting the state variable;
- and
- converting a negative reward based on the reaction of the human into a state variable set for the negative reward and outputting the state variable.
Dependencies are inferred from the claim text.