# US 10,245,726 B1 · Methods and systems for tiered programming of robotic device > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,245,726 B1 - Document ID: US10245726B1 - Status: Granted - Kind: B1 - Application number: 15/242,754 - Filed: 2016-08-22 - Granted: 2019-04-02 - Earliest claimed: 2013-06-24 - Assignees: X Development LLC - Applicants: X Development LLC - Inventors: Advait Jain; Anthony Jules; Aaron Edsinger - Web page: https://patentgrep.com/patent/US10245726B1 ## CPC - B25J 9/1661: {characterised by task planning, object-oriented languages} - B25J 9/16: Program controls (total factory control, i.e. centrally controlling a plurality of machines, G05B19/418) - B25J 9/1656: {characterised by programming, planning systems for manipulators} - G05B 19/0426: {Programming the control sequence} - G05B 19/41845: {characterised by system universality, reconfigurability, modularity} - G05B 19/41865: {characterised by job scheduling, process planning, material flow} - G06Q 10/06316: {Sequencing of tasks or work} - G05B 2219/23254: Interactive programming, sentence on screen filled in by operator - G05B 2219/31075: Modular cell elements - G05B 2219/39252: Autonomous distributed control, task distributed into each subsystem, task space - G06Q 10/06311: {Scheduling, planning or task assignment for a person or group} - Y02P 90/02: Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS] - B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50}) - G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G) - G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR - Y02P: CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS ## Abstract A method operable by a computing device is provided. The method may include receiving a request for a given task to be performed by a robotic system. The method may also determining one or more subtasks required to perform the given task, where the one or more subtasks include one or more parameters used to define the one or more subtasks. The method may also include determining an arrangement of the one or more subtasks to perform the given task, and providing for display an indication of the one or more undefined parameters for the given task. The method may also include receiving an input defining the one or more undefined parameters for the given task, and executing the one or more subtasks in the determined arrangement and in accordance with the one or more defined parameters to cause the robotic system to perform the given task. ## Claims ### Claim 1 A method operable by a computing device, the method comprising: receiving a request for a given task to be performed by a robotic system, wherein the robotic system includes a modular reconfigurable workcell including two or more modular docking bays on a surface of the workcell that support removable attachment of two or more peripherals in a fixed geometric configuration; determining two or more subtasks required to perform the given task, wherein the two or more subtasks are performed by the two or more peripherals of the robotic system, wherein a first peripheral of the two or more peripherals is configured to perform a first set of subtasks and a second peripheral of the two or more peripherals is configured to perform a second set of subtasks, wherein the first peripheral is selected based on its ability to perform the first set of subtasks, and wherein the second peripheral is selected based on its ability to perform the second set of subtasks; determining an arrangement of the two or more subtasks to perform the given task, wherein the arrangement includes a combination of the two or more subtasks in a particular order; based on the determined arrangement of the two or more subtasks, determining an arrangement of the two or more peripherals on the modular reconfigurable workcell to perform the given task; receiving an indication of attachment of the two or more peripherals to the modular reconfigurable workcell based on the determined arrangement of the two or more peripherals; and executing the two or more subtasks in the determined arrangement to cause the two or more peripherals of the robotic system to perform the given task. ### Claim 2 (depends on 1) The method of claim 1, wherein the two or more subtasks include one or more parameters used to define the two or more subtasks, wherein the one or more parameters include information required for the robotic system to execute a given subtask. ### Claim 3 (depends on 2) The method of claim 2, wherein the one or more parameters include one or more of a location of a given peripheral of the two or more peripherals, an orientation of a component of the given peripheral, a force limit for a component of the given peripheral, a rotation speed of a component of the given peripheral, a pattern for the given peripheral to follow, and a count of a number of times the given subtask is performed. ### Claim 4 (depends on 2) The method of claim 2, wherein at least one of the one or more parameters are initially defined by a default setting, wherein the default setting is based at least in part by sensor data from one or more sensors of the robotic system. ### Claim 5 (depends on 2) The method of claim 2, further comprising: based on the determined arrangement of the two or more subtasks, providing for display a visual representation of each of the two or more subtasks in the particular order simultaneously on an interface of the computing device, wherein the visual representation includes an indication of the one or more undefined parameters for the given task. ### Claim 6 (depends on 5) The method of claim 5, further comprising: receiving, via the interface of the computing device, an input defining the one or more undefined parameters for the given task, wherein a defined parameter includes information required for the robotic system to execute the given subtask. ### Claim 7 (depends on 6) The method of claim 6, wherein the input defining the one or more undefined parameters comprises moving a component of the robotic system to define a geometrical bound for the component. ### Claim 8 (depends on 1) The method of claim 1, further comprising: based on the determined arrangement of the two or more subtasks, providing for display a visual representation of each of the two or more subtasks in the particular order simultaneously on an interface of the computing device. ### Claim 9 (depends on 1) The method of claim 1, wherein the two or more subtasks include one or more error conditions, wherein an occurrence of one of the one or more error conditions cause the robotic system to stop performing the given task and provide for display an error message on an interface of a computing device. ### Claim 10 A computing device, comprising: one or more processors; and a non-transitory computer-readable medium, configured to store instructions, that when executed by the one or more processors, cause the computing device to perform functions comprising: receiving a request for a given task to be performed by a robotic system, wherein the robotic system includes a modular reconfigurable workcell including two or more modular docking bays on a surface of the workcell that support removable attachment of two or more peripherals in a fixed geometric configuration; determining two or more subtasks required to perform the given task, wherein the two or more subtasks are performed by the two or more peripherals of the robotic system, wherein a first peripheral of the two or more peripherals is configured to perform a first set of subtasks and a second peripheral of the two or more peripherals is configured to perform a second set of subtasks, wherein the first peripheral is selected based on its ability to perform the first set of subtasks, and wherein the second peripheral is selected based on its ability to perform the second set of subtasks; determining an arrangement of the two or more subtasks to perform the given task, wherein the arrangement includes a combination of the two or more subtasks in a particular order; based on the determined arrangement of the two or more subtasks, determining an arrangement of the two or more peripherals on the modular reconfigurable workcell to perform the given task; receive an indication of attachment of the two or more peripherals to the modular reconfigurable workcell based on the determined arrangement of the two or more peripherals; and executing the two or more subtasks in the determined arrangement to cause the two or more peripherals of the robotic system to perform the given task. ### Claim 11 (depends on 10) The computing device of claim 10, wherein the two or more subtasks include one or more parameters used to define the two or more subtasks, wherein the one or more parameters include information required for the robotic system to execute a given subtask. ### Claim 12 (depends on 11) The computing device of claim 11, wherein the one or more parameters include one or more of a location of a given peripheral of the two or more peripherals, an orientation of a component of the given peripheral, a force limit for a component of the given peripheral, a rotation speed of a component of the given peripheral, a pattern for the given peripheral to follow, and a count of a number of times the given subtask is performed. ### Claim 13 (depends on 11) The computing device of claim 11, wherein at least one of the one or more parameters are initially defined by a default setting, wherein the default setting is based at least in part by sensor data from one or more sensors of the robotic system. ### Claim 14 (depends on 11) The computing device of claim 11, wherein the functions further comprise: based on the determined arrangement of the two or more subtasks, providing for display a visual representation of each of the two or more subtasks in the particular order simultaneously on an interface of the computing device, wherein the visual representation includes an indication of the one or more undefined parameters for the given task; and receiving, via the interface of the computing device, an input defining the one or more undefined parameters for the given task, wherein a defined parameter includes information required for the robotic system to execute the given subtask. ### Claim 15 (depends on 14) The computing device of claim 14, wherein the input defining the one or more undefined parameters comprises moving a component of the robotic system to define a geometrical bound for the component. ### Claim 16 (depends on 10) The computing device of claim 10, wherein the functions further comprise: based on the determined arrangement of the two or more subtasks, providing for display a visual representation of each of the two or more subtasks in the particular order simultaneously on an interface of the computing device. ### Claim 17 A robotic system, comprising: a modular reconfigurable workcell; a computer-based controller in communication with the modular reconfigurable workcell, wherein the computer-based controller is configured to: receive a request for a given task to be performed by the robotic system, wherein the modular reconfigurable workcell of the robotic system includes two or more modular docking bays on a surface of the workcell that support removable attachment of two or more peripherals in a fixed geometric configuration; determine two or more subtasks required to perform the given task, wherein the two or more subtasks are performed by the two or more peripherals of the robotic system, wherein a first peripheral of the two or more peripherals is configured to perform a first set of subtasks and a second peripheral of the two or more peripherals is configured to perform a second set of subtasks, wherein the first peripheral is selected based on its ability to perform the first set of subtasks, and wherein the second peripheral is selected based on its ability to perform the second set of subtasks; determine an arrangement of the two or more subtasks to perform the given task, wherein the arrangement includes a combination of the two or more subtasks in a particular order; based on the determined arrangement of the two or more subtasks, determine an arrangement of the two or more peripherals on the modular reconfigurable workcell to perform the given task; receive an indication of attachment of the two or more peripherals to the modular reconfigurable workcell based on the determined arrangement of the two or more peripherals; and execute the two or more subtasks in the determined arrangement to cause the two or more peripherals of the robotic system to perform the given task. ### Claim 18 (depends on 17) The robotic system of claim 17, wherein respective modular docking bays include a plurality of electrical connections for a variety of power and communication busses that provide an electrical and mechanical interface between a respective peripheral and the workcell. ### Claim 19 (depends on 17) The robotic system of claim 17, wherein the two or more peripherals include two or more of a robot arm, a camera, a gripper, a seven degree of freedom (DOF) arm with a camera, a device for assessing device-under-test functionality, a sensor for measuring a location of a device-under-test, and an input/output (I/O) peripheral for interfacing to an external device. ### Claim 20 (depends on 17) The robotic system of claim 17, wherein the computer-based controller is further configured to: based on the determined arrangement of the two or more subtasks, provide for display a visual representation of each of the two or more subtasks in the particular order simultaneously on an interface of the robotic system.