US 2025/0026018 A1Published application

Method for robot assembly skill learning

AssigneeFanuc Corporation

InventorsYu Zhao, Tetsuaki Kato

Life of this patent

Estimating the term…

Loading patent…

Abstract

A method and system for robot skill learning applicable to high precision assembly tasks employing a compliance controller. An actor-critic reinforcement learning controller is coupled to the compliance controller, where an actor neural network provides target position adjustment action data to the compliance controller based on robot state data feedback, and a critic neural network is used to train the actor. The critic neural network receives the robot state data feedback and reward data from the robot, along with the action data from the actor, and correlates optimal actions associated with states in order to maximize the reward. The critic then adjusts the parameters of the actor so that the actor produces effective actions in response to the state data, leading to rapid and reliable completion of the assembly task by the compliance controller/robot system. The critic is no longer used after the actor is adequately trained.

Claims20

Dependent claims

A method for robotic assembly skill learning, said method comprising:

providing a robot controlled by a compliance controller, the robot being configured to perform an assembly operation where the robot moves a first part into an assembled position with a second part;
providing a reinforcement learning controller in communication with the compliance controller and the robot, said reinforcement learning controller having an actor module and a critic module, where the actor module includes a neural network which defines a policy for determining an action in response to state data received as feedback from the robot, and the critic module includes a neural network which optimizes a critic function correlating the state data to the action in order to maximize reward data received as feedback from the robot;
operating the robot to perform the assembly operation with the reinforcement learning controller in a learning mode, where the actor module provides the action as input to the compliance controller and the critic module updates parameters of the policy of the actor module based on the critic function, until the actor module demonstrates a predefined performance level of the assembly operation;
and
operating the robot to perform the assembly operation with the reinforcement learning controller in a production mode, where the actor module provides the action as input to the compliance controller and the critic module is no longer used.

2 more elements

A robotic assembly skill learning system, said system comprising:

a robot controlled by a compliance controller, the robot being configured to perform an assembly operation where the robot moves a first part into an assembled position with a second part;
and
a reinforcement learning controller in communication with the compliance controller and the robot, said reinforcement learning controller having an actor module and a critic module, where the actor module includes a neural network which defines a policy for determining an action in response to state data received as feedback from the robot, and the critic module includes a neural network which optimizes a critic function correlating the state data to the action in order to maximize reward data received as feedback from the robot,
where the robot performs the assembly operation with the reinforcement learning controller in a learning mode, where the actor module provides the action as input to the compliance controller and the critic module updates parameters of the policy of the actor module based on the critic function, until the actor module demonstrates a predefined performance level of the assembly operation,
and where the robot performs the assembly operation with the reinforcement learning controller in a production mode, where the actor module provides the action as input to the compliance controller and the critic module is no longer used.

2 more elements

Dependencies are inferred from the claim text.