# US 2021/0220991 A1 · Enhancing robot learning

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0220991 A1
- Document ID: US20210220991A1
- Status: Published application
- Kind: A1
- Application number: 17/222,496
- Filed: 2021-04-05
- Published: 2021-07-22
- Earliest claimed: 2017-12-27
- Assignees: X Development LLC
- Applicants: X Development LLC
- Inventors: Nareshkumar Rajkumar; Patrick Leger; Abhinav Gupta
- Web page: https://patentgrep.com/patent/US20210220991A1

## CPC

- B25J 9/163: {learning, adaptive, model based, rule based expert control}
- B25J 9/0003: {Home robots, i.e. small robots for domestic use}
- B25J 9/1653: {parameters identification, estimation, stiffness, accuracy, error analysis}
- B25J 11/008: {Manipulators for service tasks}
- G06N 3/045: Combinations of networks
- G06N 3/0499: Feedforward networks
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G06N 3/098: Distributed learning, e.g. federated learning
- G06N 20/00: Machine learning
- B25J 9/1697: {Vision controlled systems}
- G05B 2219/45108: Aid, robot for aid to, assist human disabled
- Y10S 901/01: Mobile robot
- Y10S 901/03: Teaching system
- Y10S 901/47: Optical
- B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50})
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)
- Y10S: TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS

## Abstract

Methods, systems, and apparatus, including computer-readable media storing executable instructions, for enhancing robot learning. In some implementations, a robot stores first embeddings generated using a first machine learning model, and the first embeddings include one or more first private embeddings that are not shared with other robots. The robot receives a second machine learning model from a server system over a communication network. The robot generates a second private embedding for each of the one or more first private embeddings using the second machine learning model. The robot adds the second private embeddings to the cache of the robot and removes the one or more first private embeddings from the cache of the robot.

## Claims

### Claim 1

A method comprising:
receiving, by one or more computing devices, sensor data;
generating, by the one or more computing devices, a data representation based on processing input derived from the received sensor data with a trained machine learning model;
determining, by the one or more computing devices, scores for each of multiple classifications based on respective levels of similarity between stored data representations corresponding to the classifications and the generated data representation; and
selecting, by the one or more computing devices, a classification corresponding to at least a portion of the sensor data based on the scores.

### Claim 2 (depends on 1)

The method of claim 1, wherein the receiving, generating, determining, and selecting are performed by one or more computing devices of a robot.

### Claim 3 (depends on 2)

The method of claim 2, wherein receiving the sensor data comprises capturing, by the one or more computing devices of the robot, image data using a camera of the robot.

### Claim 4 (depends on 3)

The method of claim 3, wherein the trained machine learning model is stored by the robot, and wherein the data representation is (i) an output of the trained machine learning model produced in response to processing the input or (ii) a representation of a state of the machine learning model when processing the input.

### Claim 5 (depends on 2)

The method of claim 2, comprising storing, in a data cache of the robot, records that associate the stored data representations with the classifications;
wherein selecting the classification comprises selecting the classification corresponding to a stored data representation, from among the stored data representations, that has the greatest similarity with the generated data representation.

### Claim 6 (depends on 1)

The method of claim 1, wherein the selected classification corresponding to the is a first classification; and
wherein the method comprises:
generating an output from the trained machine learning model indicating a likelihood for a second classification; and
selecting from among the first classification and a second classification.

### Claim 7 (depends on 1)

The method of claim 1, wherein the trained machine learning model is a neural network that has been trained to perform object recognition by predicting likelihoods for different object classifications;
wherein the generated data representation comprises an embedding based on activations in the neural network in response to processing the input;
wherein determining scores for each of multiple classifications comprises determining first confidence scores for object classifications based on the respective levels of similarity between the stored data representations corresponding to the classifications and the generated data representation; and
wherein selecting the classification comprises selecting a first object classification that the first confidence scores indicate has a highest likelihood of representing an object described by the sensor data;
wherein the method further includes:
obtaining output from the neural network produced in response to processing the input, the output comprising a second confidence score for each of multiple different object classifications;
selecting a second object classification that the second confidence scores in the output of the neural network indicate has a highest likelihood of representing object described by the sensor data; and
selecting, from among the first object classification and the second object classification, an object classification for the object based on the first confidence scores for the first object classification and the second confidence score for the second object classification.

### Claim 8 (depends on 1)

The method of claim 1, wherein the scores are confidence scores indicating a level of confidence in the corresponding classification or are measures of similarity between the generated data representation and the stored data representation corresponding to the classification.

### Claim 9 (depends on 1)

The method of claim 1, wherein the stored data representations are stored in a local cache of a robot, and wherein the stored data representations include data representations generated by one or more other robots using machine learning models of the one or more other robots and based on sensor data obtained by the one or more other robots.

### Claim 10 (depends on 9)

The method of claim 9, comprising:
receiving, over a communication network records indicating additional data representations and corresponding classifications generated by one or more other robots; and
storing the additional data representations and corresponding classifications in the local cache of the robot.

### Claim 11 (depends on 1)

The method of claim 1, wherein the trained machine learning model is an object recognition model.

### Claim 12 (depends on 1)

The method of claim 1, wherein the trained machine learning model is a neural network, and the generated data representation indicates activations at layer of the neural network in response to processing the input.

### Claim 13

A system comprising:
one or more computing devices; and
one or more computer-readable media storing instructions that are operable, when executed by the one or more computing devices, to cause the system to perform operations comprising:
receiving sensor data;
generating a data representation based on processing input derived from the received sensor data with a trained machine learning model;
determining scores for each of multiple classifications based on respective levels of similarity between stored data representations corresponding to the classifications and the generated data representation; and
selecting a classification corresponding to at least a portion of the sensor data based on the scores.

### Claim 14 (depends on 13)

The system of claim 13, wherein the system is a robot, and wherein the receiving, generating, determining, and selecting are performed by one or more computing devices of the robot.

### Claim 15 (depends on 14)

The system of claim 14, wherein receiving the sensor data comprises capturing, by the one or more computing devices of the robot, image data using a camera of the robot.

### Claim 16 (depends on 15)

The system of claim 15, wherein the trained machine learning model is stored by the robot, and wherein the data representation is (i) an output of the trained machine learning model produced in response to processing the input or (ii) a representation of a state of the machine learning model when processing the input.

### Claim 17 (depends on 14)

The system of claim 14, wherein the operations comprise storing, in a data cache of the robot, records that associate the stored data representations with the classifications; and
wherein selecting the classification comprises selecting the classification corresponding to a stored data representation, from among the stored data representations, that has the greatest similarity with the generated data representation.

### Claim 18 (depends on 13)

The system of claim 13, wherein the selected classification corresponding to the is a first classification;
wherein the operations comprise:
generating an output from the trained machine learning model indicating a likelihood for a second classification; and
selecting from among the first classification and a second classification.

### Claim 19 (depends on 13)

The system of claim 13, wherein the scores are confidence scores indicating a level of confidence in the corresponding classification or are measures of similarity between the generated data representation and the stored data representation corresponding to the classification.

### Claim 20

The one or more computer-readable media storing instructions that are operable, when executed by the one or more computing devices, to cause the one or more computing devices to perform operations comprising:
receiving, by the one or more computing devices, sensor data;
generating, by the one or more computing devices, a data representation based on processing input derived from the received sensor data with a trained machine learning model;
determining, by the one or more computing devices, scores for each of multiple classifications based on respective levels of similarity between stored data representations corresponding to the classifications and the generated data representation; and
selecting, by the one or more computing devices, a classification corresponding to at least a portion of the sensor data based on the scores.
