# US 2018/0021956 A1 · Robotic camera control via motion capture > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2018/0021956 A1 - Document ID: US20180021956A1 - Status: Published application - Kind: A1 - Application number: 15/216,583 - Filed: 2016-07-21 - Published: 2018-01-25 - Earliest claimed: 2016-07-21 - Assignees: Autodesk, Inc. - Applicants: Autodesk, Inc. - Inventors: Evan Patrick Atherton; David Thomasson; Maurice Ugo Conti; Heather Kerrick - Web page: https://patentgrep.com/patent/US20180021956A1 ## CPC - B25J 9/1697: {Vision controlled systems} - B25J 9/1664: {characterised by motion, path, trajectory planning} - B25J 9/1679: {characterised by the tasks executed} - B25J 9/1689: {Teleoperation} - B25J 19/023: {including video camera means} - G05D 1/0016: {characterised by the operator's input device (input arrangements for computing systems in general G06F3/00)} - G06F 3/0304: {Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042)} - G06T 7/251: {involving models} - G06V 40/23: {Recognition of whole body movements, e.g. for sport training} - H04N 23/66: Remote control of cameras or camera parts, e.g. by remote control devices - B25J 9/16: Program controls (total factory control, i.e. centrally controlling a plurality of machines, G05B19/418) - G05B 2219/35444: Gesture interface, controlled machine observes operator, executes commands - G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)} - G06T 2207/30196: Human being; Person - H04N 23/90: Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums - B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50}) - G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G) ## Abstract A motion capture setup records the movements of an operator, and a control engine then translates those movements into control signals for controlling a robot. The control engine may directly translate the operator movements into analogous movements to be performed by the robot, or the control engine may compute robot dynamics that cause a portion of the robot to mimic a corresponding portion of the operator. ## Claims ### Claim 1 A computer-implemented method for controlling a robot, the method comprising: generating motion capture data based on one or more movements of an operator; processing the motion capture data to generate control signals for controlling the robot; and transmitting the control signals to the robot to cause the robot to mimic the one or more movements of the operator. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, wherein generating the motion capture data comprises tracking at least one of head, body, and limb movements associated with the operator via a motion capture setup. ### Claim 3 (depends on 1) The computer-implemented method of claim 1, wherein processing the motion capture data to generate the control signals comprises: modeling a first set of dynamics of the operator that corresponds to the one or more movements; determining a second set of dynamics of the robot that corresponds to the one or more movements; and generating the control signals based on the second set of dynamics. ### Claim 4 (depends on 1) The computer-implemented method of claim 1, wherein causing the robot to mimic the one or more movements of the operator comprises actuating a set of joints included in the robot to perform an articulation sequence associated with the one or more movements. ### Claim 5 (depends on 4) The computer-implemented method of claim 4, wherein the robot comprises a robotic arm that includes the set of joints. ### Claim 6 (depends on 1) The computer-implemented method of claim 1, wherein causing the robot to mimic the one or more movements comprises adjusting a speed of at least one motor coupled to a propeller. ### Claim 7 (depends on 6) The computer-implemented method of claim 6, wherein the robot comprises a flying drone. ### Claim 8 (depends on 1) The computer-implemented method of claim 1, wherein causing the robot to mimic the one or more movements comprises causing only a first portion of the robot to mimic only a subset of the one or more movements. ### Claim 9 (depends on 1) The computer-implemented method of claim 1, further comprising capturing sensor data while the robot mimics the one or more movements. ### Claim 10 (depends on 1) The computer-implemented method of claim 1, wherein the one or more movements comprise a target trajectory for a sensor array coupled to the robot. ### Claim 11 A non-transitory computer-readable medium that, when executed by a processor, causes the processor to control a robot by performing the steps of: generating motion capture data based on one or more movements of an operator; processing the motion capture data to generate control signals for controlling the robot; and transmitting the control signals to the robot to cause the robot to mimic the one or more movements of the operator. ### Claim 12 (depends on 11) The non-transitory computer-readable medium of claim 11, wherein generating the motion capture data comprises tracking at least one of head, body, and limb movements associated with the operator via a motion capture setup. ### Claim 13 (depends on 11) The non-transitory computer-readable medium of claim 11, wherein processing the motion capture data to generate the control signals comprises: modeling a first set of dynamics of the operator that corresponds to the one or more movements; determining a second set of dynamics of the robot that corresponds to the one or more movements; and generating the control signals based on the second set of dynamics. ### Claim 14 (depends on 11) The non-transitory computer-readable medium of claim 11, wherein causing the robot to mimic the one or more movements of the operator comprises actuating a set of joints included in the robot to perform an articulation sequence associated with the one or more movements. ### Claim 15 (depends on 11) The non-transitory computer-readable medium of claim 11, wherein causing the robot to mimic the one or more movements comprises adjusting a speed of at least one motor coupled to a propeller. ### Claim 16 (depends on 11) The non-transitory computer-readable medium of claim 11, wherein causing the robot to mimic the one or more movements comprises causing only a first portion of the robot to mimic only a subset of the one or more movements. ### Claim 17 (depends on 11) The non-transitory computer-readable medium of claim 11, further comprising capturing sensor data while the robot mimics the one or more movements. ### Claim 18 (depends on 11) The non-transitory computer-readable medium of claim 11, wherein the one or more movements comprise a target trajectory for a sensor array coupled to the robot. ### Claim 19 A system for controlling a robot, comprising: a memory storing a control application; and a processor coupled to the memory and configured to: generate motion capture data based on one or more movements of an operator, process the motion capture data to generate control signals for controlling the robot, and transmit the control signals to the robot to cause the robot to mimic the one or more movements of the operator. ### Claim 20 (depends on 19) The system of claim 19, wherein the processor is configured to execute the control application to: generate the motion capture data; process the motion capture data to generate the control signals; and transmit the control signals to the robot.