# US 2024/0198532 A1 · Method for robot teleoperation control, robot, and electronic device > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2024/0198532 A1 - Document ID: US20240198532A1 - Status: Published application - Kind: A1 - Application number: 18/518,960 - Filed: 2023-11-25 - Published: 2024-06-20 - Earliest claimed: 2022-12-20 - Assignees: Ubtech Robotics Corp Ltd - Applicants: Ubtech Robotics Corp Ltd - Inventors: MENG YAN; Chunyu Chen; Jiangchen Zhou; Lihua Lu - Web page: https://patentgrep.com/patent/US20240198532A1 ## CPC - B25J 9/1689: {Teleoperation} - B25J 9/16: Program controls (total factory control, i.e. centrally controlling a plurality of machines, G05B19/418) - B25J 9/163: {learning, adaptive, model based, rule based expert control} - Y02P 90/02: Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS] - B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50}) - Y02P: CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS ## Abstract A method for robot telcoperation control is provided. The method includes acquiring target action data and displacement data of a target object, wherein the target action data includes head action data and arm action data; controlling a target robot to act according to the target action data to enable the target robot to complete an action corresponding to the target action data; and performing centroid trajectory planning on the target robot based on a model predictive control (MPC) algorithm according to the displacement data to obtain a target centroid trajectory, and establishing a spring-damping system to track the target centroid trajectory so as to enable the target robot to move to a position corresponding to the displacement data. ## Claims ### Claim 1 A method for robot teleoperation control, comprising: acquiring target action data and displacement data of a target object, wherein the target action data includes head action data and arm action data; controlling a target robot to act according to the target action data to enable the target robot to complete an action corresponding to the target action data; and performing centroid trajectory planning on the target robot based on a model predictive control (MPC) algorithm according to the displacement data to obtain a target centroid trajectory, and establishing a spring-damping system to track the target centroid trajectory so as to enable the target robot to move to a position corresponding to the displacement data. ### Claim 2 (depends on 1) The method according to claim 1, wherein before acquiring the target action data of the target object, the method further includes: acquiring head calibration action data, arm calibration action data and calibration position data of the target object; controlling a head action of the target robot according to the head calibration action data to enable a head of the target robot to complete the head action corresponding to the head calibration action data; controlling an arm action of the target robot according to the arm calibration action data to enable arms of the target robot to complete the arm action corresponding to the arm calibration action data; establishing a human body coordinate system with the calibration position data as an origin; and establishing a robot coordinate system with calibration position data of the target robot as an origin. ### Claim 3 (depends on 1) The method according to claim 1, wherein acquiring the displacement data of the target object includes: acquiring motion posture data and skeleton data of the target object; calculating a joint rotation matrix of the target object according to a data fusion algorithm, a filtering algorithm and the motion posture data; calculating a skeleton vector of the target object according to the skeleton data; and calculating a product of the joint rotation matrix and the skeleton vector to obtain the displacement data. ### Claim 4 (depends on 1) The method according to claim 1, wherein controlling the target robot to act according to the target action data includes: controlling the target robot to act according to the head action data to enable a head of the target robot to complete an action corresponding to the head action data; and controlling the target robot to act according to the arm action data to enable arms of the target robot to complete an action corresponding to the arm action data. ### Claim 5 (depends on 1) The method according to claim 1, wherein performing centroid trajectory planning on the target robot based on the MPC algorithm according to the displacement data to obtain the target centroid trajectory and establishing the spring-damping system to track the target centroid trajectory so as to enable the target robot to move to the position corresponding to the displacement data includes: mapping the displacement data into a human body coordinate system to obtain mapped displacement data; mapping the mapped displacement data into a robot coordinate system to obtain the target centroid trajectory; and controlling the target robot to act according to the target centroid trajectory to enable the target robot to move to the position corresponding to the displacement data. ### Claim 6 (depends on 1) The method according to claim 1, further comprising: controlling a head camera of the target robot to photograph to obtain field of view data; and sending the field of view data to a target device. ### Claim 7 (depends on 6) The method according to claim 6, further comprising: obtaining information about an environment where the target robot is located according to the field of view data; and adjusting the target robot according to the information. ### Claim 8 A robot, comprising: a memory storing a computer program; and a processor couple to the memory, wherein the processor is configured to execute the computer program to: acquire target action data and displacement data of a target object, wherein the target action data includes head action data and arm action data; perform a target action according to the target action data; and perform centroid trajectory planning on the target robot based on a model predictive control (MPC) algorithm according to the displacement data to obtain a target centroid trajectory, establish a spring-damping system, and move to a position corresponding to the displacement data according to the target centroid trajectory. ### Claim 9 (depends on 8) The robot according to claim 8, wherein the processor is further configured to: acquire head calibration action data, arm calibration action data and calibration position data of the target object before acquiring the target action data of the target object; control a head action of the robot according to the head calibration action data to enable a head of the robot to complete the head action corresponding to the head calibration action data, and control an arm action of the robot according to the arm calibration action data to enable arms of the robot to complete the arm action corresponding to the arm calibration action data; establish a human body coordinate system with the calibration position data as an origin; and establish a robot coordinate system with calibration position data of the target robot as an origin. ### Claim 10 (depends on 8) The robot according to claim 8, wherein the processor is further configured to: acquire motion posture data and skeleton data of the target object; calculate a joint rotation matrix of the target object according to a data fusion algorithm, a filtering algorithm and the motion posture data; calculate a skeleton vector of the target object according to the skeleton data; and calculate a product of the joint rotation matrix and the skeleton vector to obtain the displacement data. ### Claim 11 (depends on 8) The robot according to claim 8, wherein the processor is further configured to: control a head action according to the head action data to enable the head to complete the head action corresponding to the head action data; and control an arm action according to the arm action data to enable the arm to complete the arm action corresponding to the arm action data. ### Claim 12 (depends on 8) The robot according to claim 8, wherein the processor is further configured to: map the displacement data into a human body coordinate system to obtain mapped displacement data; map the mapped displacement data into a robot coordinate system to obtain the target centroid trajectory; and move to a target position corresponding to the displacement data according to the target centroid trajectory. ### Claim 13 (depends on 8) The robot according to claim 8, wherein the processor is further configured to: control a head camera to photograph to obtain field of view data; and send the field of view data to a target device. ### Claim 14 An electronic device, comprising: a memory storing a computer program; a processor coupled to the memory, wherein the computer program when executed by the processor causes the processor to perform a method for robot teleoperation control; wherein the method includes: acquiring target action data and displacement data of a target object, wherein the target action data includes head action data and arm action data; controlling a target robot to act according to the target action data to enable the target robot to complete an action corresponding to the target action data; and performing centroid trajectory planning on the target robot based on a model predictive control (MPC) algorithm according to the displacement data to obtain a target centroid trajectory, and establishing a spring-damping system to track the target centroid trajectory so as to enable the target robot to move to a position corresponding to the displacement data. ### Claim 15 (depends on 14) The electronic device according to claim 14, wherein before acquiring the target action data of the target object, the method further includes: acquiring head calibration action data, arm calibration action data and calibration position data of the target object; controlling a head action of the target robot according to the head calibration action data to enable a head of the target robot to complete the head action corresponding to the head calibration action data; controlling an arm action of the target robot according to the arm calibration action data to enable arms of the target robot to complete the arm action corresponding to the arm calibration action data; establishing a human body coordinate system with the calibration position data as an origin; and establishing a robot coordinate system with calibration position data of the target robot as an origin. ### Claim 16 (depends on 14) The electronic device according to claim 14, wherein acquiring the displacement data of the target object includes: acquiring motion posture data and skeleton data of the target object; calculating a joint rotation matrix of the target object according to a data fusion algorithm, a filtering algorithm and the motion posture data; calculating a skeleton vector of the target object according to the skeleton data; and calculating a product of the joint rotation matrix and the skeleton vector to obtain the displacement data. ### Claim 17 (depends on 14) The electronic device according to claim 14, wherein controlling the target robot to act according to the target action data includes: controlling the target robot to act according to the head action data to enable a head of the target robot to complete an action corresponding to the head action data; and controlling the target robot to act according to the arm action data to enable arms of the target robot to complete an action corresponding to the arm action data. ### Claim 18 (depends on 14) The electronic device according to claim 14, wherein performing centroid trajectory planning on the target robot based on the MPC algorithm according to the displacement data to obtain the target centroid trajectory and establishing the spring-damping system to track the target centroid trajectory so as to enable the target robot to move to the position corresponding to the displacement data includes: mapping the displacement data into a human body coordinate system to obtain mapped displacement data; mapping the mapped displacement data into a robot coordinate system to obtain the target centroid trajectory; and controlling the target robot to act according to the target centroid trajectory to enable the target robot to move to the position corresponding to the displacement data. ### Claim 19 (depends on 14) The electronic device according to claim 14, wherein the method further includes: controlling a head camera of the target robot to photograph to obtain field of view data; and sending the field of view data to a target device. ### Claim 20 (depends on 19) The electronic device according to claim 19, wherein the method further includes: obtaining information about an environment where the target robot is located according to the field of view data; and adjusting the target robot according to the information.