# US 11,305,428 B2 · Robot motion planning device, robotic system, and method > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,305,428 B2 - Document ID: US11305428B2 - Status: Granted - Kind: B2 - Application number: 16/567,033 - Filed: 2019-09-11 - Published: 2020-05-28 - Granted: 2022-04-19 - Earliest claimed: 2018-11-28 - Assignees: Kabushiki Kaisha Toshiba - Applicants: Kabushiki Kaisha Toshiba - Inventors: Toshimitsu Kaneko; Tatsuya Tanaka; Ryosuke Nonaka; Nao Mishima; Tatsuo Kozakaya - Web page: https://patentgrep.com/patent/US11305428B2 ## CPC - B25J 9/1633: {compliant, force, torque control, e.g. combined with position control} - B25J 9/1666: {Avoiding collision or forbidden zones} - G05B 2219/39322: Force and position control - G05B 2219/40499: Reinforcement learning algorithm - B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50}) - G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G) ## Abstract According to one embodiment, a robot motion planning device includes processing circuitry. The processing circuitry receives observation information obtained by observing at least part of a movable range of a robot. The processing circuitry determines, in a case where first observation information is received, a target position to which the robot is to make a motion, using an action-value function and the first observation information. The processing circuitry receives measurement information obtained by measuring a state of the robot, calculates a difference corresponding to the first observation information, using the measurement information, and determines a motion plan of a force-controlled motion of the robot, based on the target position and the difference. ## Claims ### Claim 1 A robot motion planning device comprising: a memory; and processing circuitry configured to: receive observation information obtained by observing at least part of a movable range of a robot; determine, in a case where first observation information is received, a first target position to which the robot is to make a motion, using an action-value function and the first observation information, the action-value function being a function of observation information and a position and being indicative of a value of a motion to be made by the robot; receive measurement information obtained by measuring a state of the robot; calculate a first difference corresponding to the first observation information, using the measurement information; and determine a motion plan of a force-controlled motion of the robot, based on the first target position and the first difference, wherein the processing circuitry is configured to make a distance between a start point and an end point of the force-controlled motion longer as the first difference is greater. ### Claim 2 (depends on 1) The robot motion planning device of claim 1, wherein the processing circuitry is further configured to: determine a first reward, based on a motion made by the robot according to the motion plan; and update the action-value function, using the first reward. ### Claim 3 (depends on 2) The robot motion planning device of claim 2, wherein the processing circuitry is configured to: calculate an update amount of the action-value function, using at least the first reward and the action-value function; and update the action-value function, using the update amount. ### Claim 4 (depends on 2) The robot motion planning device of claim 2, wherein the processing circuitry is further configured to: determine second rewards corresponding to positions, respectively, based on the motion made by the robot according to the motion plan; and update the action-value function, using the second rewards. ### Claim 5 (depends on 2) The robot motion planning device of claim 2, wherein the processing circuitry is further configured to determine the first target position at random from a range that is determined based on the action-value function and the first difference. ### Claim 6 (depends on 1) The robot motion planning device of claim 1, wherein the processing circuitry is configured to: receive first measurement information obtained by measuring a state of the robot that makes a motion according to the motion plan; and update the first difference, using the first measurement information. ### Claim 7 (depends on 6) The robot motion planning device of claim 6, wherein the processing circuitry is configured to update the first difference, using a difference between a first change point of a first action-value function of the action-value function and a second change point of the first measurement information, the first action-value function corresponding to the first observation information. ### Claim 8 (depends on 7) The robot motion planning device of claim 7, wherein the first measurement information comprises pieces of information, and each of the pieces of information indicates a position of a movable portion of the robot and force applied to the movable portion at the position, and the processing circuitry is configured to update the first difference, using a difference between a first position where the first action-value function declines by a first threshold value or more and a second position where the first measurement information exceeds a second threshold value. ### Claim 9 (depends on 1) The robot motion planning device of claim 1, wherein the processing circuitry is further configured to display information on the first difference on a screen of a display. ### Claim 10 (depends on 9) The robot motion planning device of claim 9, wherein the information on the first difference indicates whether the robot is abnormal based on a fluctuation in the first difference in chronological order. ### Claim 11 (depends on 1) The robot motion planning device of claim 1, wherein the processing circuitry is further configured to display information on one or more differences on a screen of a display device, the one or more differences corresponding to one or more pieces of observation information that are received, respectively. ### Claim 12 (depends on 11) The robot motion planning device of claim 11, wherein the processing circuitry is further configured to display a statistic of the one or more differences of each type of object on the screen as the information on the one or more differences. ### Claim 13 (depends on 1) The robot motion planning device of claim 1, wherein the processing circuitry is configured to: acquire a first action-value function corresponding to the first observation information from the action-value function; and determine the first target position, using the first action-value function. ### Claim 14 (depends on 13) The robot motion planning device of claim 13, wherein the processing circuitry is configured to determine a position where the first action-value function is greatest as the first target position. ### Claim 15 (depends on 1) The robot motion planning device of claim 1, further comprising: a transmitter configured to transmit a control signal based on the motion plan to the robot. ### Claim 16 (depends on 15) The robot motion planning device of claim 15, wherein the processing circuitry is further configured to: receive first measurement information obtained by measuring a state of the robot while the robot is making a motion based on the control signal; and update the first difference, using the first measurement information. ### Claim 17 A robotic system comprising: a robot; and a robot motion planning device comprising: a memory; and processing circuitry configured to: receive observation information obtained by observing at least part of a movable range of a robot; determine, in a case where first observation information is received, a first target position to which the robot is to make a motion, using an action-value function and the first observation information, the action-value function being a function of observation information and a position and being indicative of a value of a motion to be made by the robot; receive measurement information obtained by measuring a state of the robot; calculate a first difference corresponding to the first observation information, using the measurement information; and determine a motion plan of a force-controlled motion of the robot, based on the first target position and the first difference, wherein the processing circuitry is configured to make a distance between a start point and an end point of the force-controlled motion longer as the first difference is greater. ### Claim 18 A method comprising: receiving observation information obtained by observing at least part of a movable range of a robot; in a case where first observation information is received, determining a first target position to which the robot is to make a motion, using an action-value function and the first observation information, the action-value function being a function of observation information and a position and being indicative of a value of a motion to be made by the robot; receiving measurement information obtained by measuring a state of the robot; calculating a first difference corresponding to the first observation information, using the measurement information; and determining a motion plan of a force-controlled motion of the robot, based on the first target position and the first difference, wherein the determining the motion plan comprises making a distance between a start point and an end point of the force-controlled motion longer as the first difference is greater.