# US 12,214,507 B1 · Injecting noise into robot simulation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,214,507 B1
- Document ID: US12214507B1
- Status: Granted
- Kind: B1
- Application number: 18/381,786
- Filed: 2023-10-19
- Granted: 2025-02-04
- Earliest claimed: 2021-06-02
- Assignees: Google Inc.
- Applicants: Google LLC
- Inventors: Matthew Bennice; Paul Bechard; Joséphine Simon; Chuyuan Fu; Wenlong Lu
- Web page: https://patentgrep.com/patent/US12214507B1

## CPC

- B25J 9/1605: {Simulation of manipulator lay-out, design, modelling of manipulator}
- B25J 9/1671: {characterised by simulation, either to verify existing program or to create and verify new program, CAD/CAM oriented, graphic oriented programming systems}
- B25J 9/163: {learning, adaptive, model based, rule based expert control}
- B25J 9/1638: {compensation for arm bending/inertia, pay load weight/inertia}
- B25J 9/1697: {Vision controlled systems}
- B25J 13/085: {Force or torque sensors (B25J13/082, B25J13/084 take precedence)}
- B25J 13/088: {with position, velocity or acceleration sensors}
- B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50})

## Abstract

Implementations are provided for increasing realism of robot simulation by injecting noise into various aspects of the robot simulation. In various implementations, a three-dimensional (3D) environment may be simulated and may include a simulated robot controlled by an external robot controller. Joint command(s) issued by the robot controller and/or simulated sensor data passed to the robot controller may be intercepted. Noise may be injected into the joint command(s) to generate noisy commands. Additionally or alternatively, noise may be injected into the simulated sensor data to generate noisy sensor data. Joint(s) of the simulated robot may be operated in the simulated 3D environment based on the one or more noisy commands. Additionally or alternatively, the noisy sensor data may be provided to the robot controller to cause the robot controller to generate joint commands to control the simulated robot in the simulated 3D environment.

## Claims

### Claim 1

A method implemented using one or more processors, comprising:
simulating an environment, wherein the simulated environment includes a simulated robot controlled by an external robot controller;
intercepting a temporal sequence of robot commands issued by the robot controller to control the simulated robot in the simulated environment;
generating a temporal sequence of progressing noise values using a time series machine learning model, wherein the time series machine learning model is trained based on empirical data about effects of environmental conditions on robot joint operation;
injecting the temporal sequence of progressing noise values into corresponding robot commands of the temporal sequence of robot commands to generate noisy robot commands, wherein the injecting includes altering, skewing, biasing, or replacing existing robot command values with noisy robot command values; and
operating the simulated robot in the simulated environment based on the noisy robot commands.

### Claim 2 (depends on 1)

The method of claim 1, wherein injecting the temporal sequence of progressing noise values includes:
operating a state machine to generate the temporal sequence of progressing noise values, wherein one or more progressing noise values of the temporal sequence of progressing noise values are generated based at least in part on one or more previous progressing noise values of the temporal sequence of progressing noise values.

### Claim 3 (depends on 1)

The method of claim 1, further comprising generating the temporal sequence of progressing noise values using a time series machine learning model.

### Claim 4 (depends on 3)

The method of claim 3, wherein the generating comprises iteratively applying time series data indicative of environmental conditions as inputs across the time series machine learning model.

### Claim 5

A method implemented using one or more processors, comprising:
simulating an environment, wherein the simulated environment includes a simulated robot controlled by an external robot controller;
intercepting a temporal sequence of robot commands issued by the robot controller to control the simulated robot in the simulated environment;
generating a temporal sequence of progressing noise values using a time series machine learning model, wherein the generating comprises iteratively applying time series data indicative of wear and tear metrics associated with a real-world robot;
injecting the temporal sequence of progressing noise values into corresponding robot commands of the temporal sequence of robot commands to generate noisy robot commands, wherein the injecting includes altering, skewing, biasing, or replacing existing robot command values with noisy robot command values; and
operating the simulated robot in the simulated environment based on the noisy robot commands.

### Claim 6

A method implemented using one or more processors, comprising:
simulating an environment, wherein the simulated environment includes a simulated robot controlled by an external robot controller;
intercepting a temporal sequence of robot commands issued by the robot controller to control the simulated robot in the simulated environment;
injecting a temporal sequence of progressing noise values into corresponding robot commands of the temporal sequence of robot commands to generate noisy robot commands, wherein the injecting includes altering, skewing, biasing, or replacing existing robot command values with noisy robot command values, and wherein the temporal sequence of progressing noise values simulate a bias of gravity over time on oscillations of one or more joints of the simulated robot; and
operating the simulated robot in the simulated environment based on the noisy robot commands.

### Claim 7 (depends on 1)

The method of claim 1, wherein the temporal sequence of progressing noise values are calculated based on one or more operating tolerances of a robot controlled by the robot controller.

### Claim 8 (depends on 7)

The method of claim 7, wherein one or more of the operating tolerances include a motor operating tolerance.

### Claim 9

A system comprising one or more processors and memory storing instructions that, in response to execution by the one or more processors, cause the one or more processors to:
simulate an environment, wherein the simulated environment includes a simulated robot controlled by an external robot controller;
intercept a temporal sequence of robot commands issued by the robot controller to control the simulated robot in the simulated environment;
generate a temporal sequence of progressing noise values using a time series machine learning model, wherein the time series machine learning model is trained based on empirical data about effects of environmental conditions on robot joint operation;
inject the temporal sequence of progressing noise values into corresponding robot commands of the temporal sequence of robot commands to generate noisy robot commands, wherein the injecting includes altering, skewing, biasing, or replacing existing robot command values with noisy robot command values; and
operate the simulated robot in the simulated environment based on the noisy robot commands.

### Claim 10 (depends on 9)

The system of claim 9, wherein the instructions to inject the temporal sequence of progressing noise values include instructions to:
operate a state machine to generate the temporal sequence of progressing noise values, wherein one or more progressing noise values of the temporal sequence of progressing noise values are generated based at least in part on one or more previous progressing noise values of the temporal sequence of progressing noise values.

### Claim 11 (depends on 9)

The system of claim 9, further comprising instructions to generate the temporal sequence of progressing noise values using a time series machine learning model.

### Claim 12 (depends on 11)

The system of claim 11, wherein the instructions to generate include instructions to iteratively apply time series data indicative of environmental conditions as inputs across the time series machine learning model.

### Claim 13

A system comprising one or more processors and memory storing instructions that, in response to execution by the one or more processors, cause the one or more processors to:
simulate an environment, wherein the simulated environment includes a simulated robot controlled by an external robot controller;
intercept a temporal sequence of robot commands issued by the robot controller to control the simulated robot in the simulated environment;
generate a temporal sequence of progressing noise values using a time series machine learning model, wherein the instructions to generate include instructions to iteratively apply time series data indicative of wear and tear metrics associated with a real-world robot;
inject the temporal sequence of progressing noise values into corresponding robot commands of the temporal sequence of robot commands to generate noisy robot commands, wherein the injecting includes altering, skewing, biasing, or replacing existing robot command values with noisy robot command values; and
operate the simulated robot in the simulated environment based on the noisy robot commands.

### Claim 14

A system comprising one or more processors and memory storing instructions that, in response to execution by the one or more processors, cause the one or more processors to:
simulate an environment, wherein the simulated environment includes a simulated robot controlled by an external robot controller;
intercept a temporal sequence of robot commands issued by the robot controller to control the simulated robot in the simulated environment;
inject a temporal sequence of progressing noise values into corresponding robot commands of the temporal sequence of robot commands to generate noisy robot commands, wherein the injecting includes altering, skewing, biasing, or replacing existing robot command values with noisy robot command values, and wherein the temporal sequence of progressing noise values simulate a bias of gravity over time on oscillations of one or more joints of the simulated robot; and
operate the simulated robot in the simulated environment based on the noisy robot commands.

### Claim 15 (depends on 9)

The system of claim 9, wherein the temporal sequence of progressing noise values are calculated based on one or more operating tolerances of a robot controlled by the robot controller.

### Claim 16 (depends on 15)

The system of claim 15, wherein one or more of the operating tolerances include a motor operating tolerance.
