# US 11,992,948 B2 · Real-time robotics control framework > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,992,948 B2 - Document ID: US11992948B2 - Status: Granted - Kind: B2 - Application number: 17/246,002 - Filed: 2021-04-30 - Published: 2022-11-03 - Granted: 2024-05-28 - Earliest claimed: 2021-04-30 - Assignees: Intrinsic Innovation LLC - Applicants: Intrinsic Innovation LLC - Inventors: Andre Gaschler; Gregory J. Prisament; Sean Alexander Cassero; Nicholas Julian Cox; Benjamin Bremer; Nils Berg; Michael Beardsworth - Web page: https://patentgrep.com/patent/US11992948B2 ## CPC - B25J 9/1658: {characterised by programming language} - B25J 9/1628: {characterised by the control loop} - B25J 9/1664: {characterised by motion, path, trajectory planning} - H04L 65/65: Network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP] - H04L 69/329: in the application layer [OSI layer 7] - G05B 2219/40512: Real time path planning, trajectory generation - H04L 67/12: specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks - H04L 69/321: Interlayer communication protocols or service data unit [SDU] definitions; Interfaces between layers - B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50}) - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G) ## Abstract Methods, systems, and apparatus, including computer programs encoded on computer storage media, for controlling a robot to perform a custom real-time action that uses streaming inputs. One of the methods comprises receiving a definition of a custom real-time streaming control function that defines a custom streaming action, wherein the custom streaming action specifies a goal state for a robot in an operating environment; providing a command to initiate the custom streaming action; and repeatedly providing updated goal states for the custom streaming action, wherein the control layer of the framework is configured to execute the custom streaming action including driving the robot toward a most recent goal state at each tick of a real-time robotics control cycle. ## Claims ### Claim 1 A computer-implemented method comprising: receiving, by an application layer of a real-time robotics control framework, a definition of a custom real-time streaming control function that defines a custom streaming action, wherein the custom streaming action specifies a goal state for a robot in an operating environment; providing, by the application layer to a control layer of the framework, a command to initiate the custom streaming action; and repeatedly providing, by the application layer to the control layer of the framework and at a slower rate than a timing requirement of a real-time robotics control cycle, updated goal states for the custom streaming action, wherein the application layer repeatedly executes code of the custom real-time streaming control function to generate the updated goal states, wherein the control layer of the framework executes the custom streaming action to drive the robot toward a most recently received goal state at each predetermined tick of the real-time robotics control cycle, and wherein the control layer comprises a non-real-time thread and a real-time thread, wherein the non-real-time thread is configured to (i) communicate with the application layer, (ii) receive an updated goal state from the application layer, and (iii) select a future tick of the real-time robotics control cycle at which to provide the updated goal state to the real-time thread, and wherein the real-time thread is configured to drive the robot according to the real-time robotics control cycle. ### Claim 2 (depends on 1) The method of claim 1, wherein the control layer is configured to drive the robot toward the most recent goal state during one or more additional ticks of the real-time robotics control cycle before receiving any additional updated goal states from the application layer. ### Claim 3 (depends on 1) The method of claim 1, wherein the rate at which the application layer provides the updated goal states is a user-defined time period specified in the custom real-time streaming control function. ### Claim 4 (depends on 1) The method of claim 1, wherein the custom real-time streaming control function specifies a user-selected interpolation function to compute goal states. ### Claim 5 (depends on 4) The method of claim 4, wherein the user-selected interpolation function is a custom interpolation function defined by the custom real-time streaming control function. ### Claim 6 (depends on 4) The method of claim 4, wherein the user-selected interpolation function is selected from a library of interpolation functions provided by the framework. ### Claim 7 (depends on 1) The method of claim 1, wherein the goal state comprises a user-specified goal position and a user-specified goal velocity. ### Claim 8 A system comprising one or more computers and one or more storage devices storing instructions that when executed by the one or more computers cause the one or more computers to perform operations comprising: receiving, by an application layer of a real-time robotics control framework, a definition of a custom real-time streaming control function that defines a custom streaming action, wherein the custom streaming action specifies a goal state for a robot in an operating environment; providing, by the application layer to a control layer of the framework, a command to initiate the custom streaming action; and repeatedly providing, by the application layer to the control layer of the framework and at a slower rate than a timing requirement of a real-time robotics control cycle, updated goal states for the custom streaming action, wherein the application layer repeatedly executes code of the custom real-time streaming control function to generate the updated goal states, wherein the control layer of the framework executes the custom streaming action to drive the robot toward a most recently received goal state at each predetermined tick of the real-time robotics control cycle, and wherein the control layer comprises a non-real-time thread and a real-time thread, wherein the non-real-time thread is configured to (i) communicate with the application layer, (ii) receive an updated goal state from the application layer, and (iii) select a future tick of the real-time robotics control cycle at which to provide the updated goal state to the real-time thread, and wherein the real-time thread is configured to drive the robot according to the real-time robotics control cycle. ### Claim 9 (depends on 8) The system of claim 8, wherein the control layer is configured to drive the robot toward the most recent goal state during one or more additional ticks of the real-time robotics control cycle before receiving any additional updated goal states from the application layer. ### Claim 10 (depends on 8) The system of claim 8, wherein the rate at which the application layer provides the updated goal states is a user-defined time period specified in the custom real-time streaming control function. ### Claim 11 (depends on 8) The system of claim 8, wherein the custom real-time streaming control function specifies a user-selected interpolation function to compute goal states. ### Claim 12 (depends on 11) The system of claim 11, wherein the user-selected interpolation function is a custom interpolation function defined by the custom real-time streaming control function. ### Claim 13 (depends on 11) The system of claim 11, wherein the user-selected interpolation function is selected from a library of interpolation functions provided by the framework. ### Claim 14 (depends on 8) The system of claim 8, wherein the goal state comprises a user-specified goal position and a user-specified goal velocity. ### Claim 15 One or more non-transitory computer-readable storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations comprising: receiving, by an application layer of a real-time robotics control framework, a definition of a custom real-time streaming control function that defines a custom streaming action, wherein the custom streaming action specifies a goal state for a robot in an operating environment; providing, by the application layer to a control layer of the framework, a command to initiate the custom streaming action; and repeatedly providing, by the application layer to the control layer of the framework and at a slower rate than a timing requirement of a real-time robotics control cycle, updated goal states for the custom streaming action, wherein the application layer repeatedly executes code of the custom real-time streaming control function to generate the updated goal states, wherein the control layer of the framework executes the custom streaming action to drive the robot toward a most recently received goal state at each predetermined tick of the real-time robotics control cycle, and wherein the control layer comprises a non-real-time thread and a real-time thread, wherein the non-real-time thread is configured to (i) communicate with the application layer, (ii) receive an updated goal state from the application layer, and (iii) select a future tick of the real-time robotics control cycle at which to provide the updated goal state to the real-time thread, wherein the real-time thread is configured to drive the robot according to the real-time robotics control cycle.