# US 12,632,695 B2 · Robotic demonstration retrieval systems and methods

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,632,695 B2
- Document ID: US12632695B2
- Status: Granted
- Kind: B2
- Application number: 17/351,351
- Filed: 2021-06-18
- Published: 2022-12-22
- Granted: 2026-05-19
- Earliest claimed: 2021-06-18
- Assignees: Naver Corporation
- Applicants: Naver Corporation
- Inventors: Julien Perez; Theo Cachet
- Web page: https://patentgrep.com/patent/US12632695B2

## CPC

- B25J 9/1664: {characterised by motion, path, trajectory planning}
- G06N 3/04: Architecture, e.g. interconnection topology
- G05B 19/42: Recording and playback systems, i.e. in which the program is recorded from a cycle of operations, e.g. the cycle of operations being manually controlled, after which this record is played back on the same machine
- G05B 2219/40391: Human to robot skill transfer
- G05B 2219/50391: Robot
- B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50})
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)

## Abstract

A robot system includes a selection module configured to select a stored demonstration for a robot from a database of stored demonstrations for different tasks of the robot; an encoder module of an attention model, the encoder module configured to determine a similarity value reflecting a similarity between: a user input demonstration for the robot; and the stored demonstration for the robot; and an indicator module configured to indicate whether the stored demonstration is the same as the user input demonstration and belongs to the same task based on the similarity value.

## Claims

### Claim 1

A robot system comprising:
one or more processors; and
memory including code that, when executed by the one or more processors, performs to:
select a stored demonstration for a robot from a database of stored demonstrations for different tasks of the robot;
using an encoder of an attention model, determine a similarity value reflecting a similarity between:
a user input demonstration for the robot; and
the selected one of the stored demonstrations for the robot;
wherein the similarity value is determined based on masking one or more scaled outputs and generating a concatenation based on the user input demonstration and the selected one of the stored demonstrations;
indicate whether the selected one of the stored demonstrations corresponds to the user input demonstration and belongs to the same task based on the similarity value; and
actuate actuators of the robot based on a policy trained using the stored demonstration output for augmenting training of the policy,
wherein the encoder is trained using a training dataset of training demonstrations and a listwise loss over concatenated pairs of training demonstrations, wherein each of the concatenated pairs of training demonstrations are based on a task identifier associated with each training demonstration,
the training demonstrations each including a temporal sequence of observation-action tuples of successive behaviors for tasks.

### Claim 2 (depends on 1)

The robot system of claim 1 wherein the code, when executed by the one or more processors, performs to indicate that the selected one of the stored demonstrations corresponds to the user input demonstration and belongs to the same task when the similarity value is greater than a predetermined value.

### Claim 3 (depends on 2)

The robot system of claim 2 wherein the code, when executed by the one or more processors, performs to output the selected one of the stored demonstrations for augmenting training of the policy for the task in response to a determination that the user input demonstration belongs to the same task as the selected one of the stored demonstration.

### Claim 4 (depends on 2)

The robot system of claim 2 wherein the code, when executed by the one or more processors, performs to indicate that the selected one of the stored demonstrations does not correspond to the user input demonstration when the similarity value is less than the predetermined value.

### Claim 5 (depends on 1)

The robot system of claim 1 wherein:
the selected one of the stored demonstration includes a first time series of at least 6 degree of freedom (DoF) poses of an end effector of the robot; and
the user input demonstration includes a second time series of at least 6 DoF poses of an end effector of the robot.

### Claim 6 (depends on 1)

The robot system of claim 1 wherein the encoder includes a stack of 6 identical layers.

### Claim 7 (depends on 6)

The robot system of claim 6 wherein each of the six identical layers includes a first sub-layer and a second sub-layer.

### Claim 8 (depends on 7)

The robot system of claim 7 wherein the first sublayers perform multi-head attention.

### Claim 9 (depends on 8)

The robot system of claim 8 wherein the multi-head attention includes performing scaled dot product attention.

### Claim 10 (depends on 7)

The robot system of claim 7 wherein the second sublayers define a position wise fully connected feed forward network.

### Claim 11 (depends on 7)

The robot system of claim 7 wherein the six identical layers include residual connections around the first and second sub-layers.

### Claim 12 (depends on 1)

The robot system of claim 1 wherein the code, when executed by the one or more processors, performs to:
select a second one of the stored demonstrations for the robot from the database of stored demonstrations in response to a determination that the selected one of the stored demonstrations does not correspond to the user input demonstration;
determine a second similarity value reflecting a second similarity between:
the user input demonstration for the robot; and
the second one of the stored demonstrations for the robot; and
indicate whether the second one of the stored demonstrations corresponds to the user input demonstration and belongs to the same task based on the second similarity value.

### Claim 13 (depends on 1)

The robot system of claim 1 wherein the encoder has a Transformer architecture.

### Claim 14 (depends on 1)

The robot system of claim 1 wherein the code, when executed by the one or more processors, performs to determine one or more other ones of the stored demonstrations associated with the selected one of the stored demonstrations in response to the determination that the one or more other stored demonstrations corresponds to the user input demonstration and belong to the same task.

### Claim 15 (depends on 14)

The robot system of claim 14 wherein the code, when executed by the one or more processors, further performs to actuate actuators of the robot based on the policy trained using the one or more other ones of the stored demonstrations in response to a determination that the one or more other ones of the stored demonstrations are the same as the user input demonstration and belong to the same task.

### Claim 16

A method for operating a robot system, the method comprising:
selecting a stored demonstration for a robot from a database of stored demonstrations for different tasks of the robot;
determining, using an attention model, a similarity value reflecting a similarity between:
a user input demonstration for the robot; and
one or more of the stored demonstrations for the robot;
wherein the determining the similarity value includes determining the similarity value based on masking one or more scaled outputs and generating a concatenation based on the user input demonstration and the selected one of the stored demonstrations;
producing an output indicating whether the one or more stored demonstrations correspond to as the user input demonstration and belong to the same task based on the similarity value; and
actuating actuators of the robot based on a policy trained using the selected one of the stored demonstrations output for augmenting training of the policy,
wherein the attention model is trained using a training dataset of training demonstrations and a listwise loss over concatenated pairs of training demonstrations, wherein each of the concatenated pairs of training demonstrations are based on a task identifier associated with each training demonstration,
the training demonstrations each including a temporal sequence of observation-action tuples of successive behaviors for tasks.

### Claim 17 (depends on 16)

The method of claim 16 wherein the producing the output further includes producing an output that indicates that the one or more stored demonstrations correspond to the user input demonstration and belong to the same task when their similarity value is greater than a predetermined value.

### Claim 18 (depends on 17)

The method of claim 17 further comprising outputting the one or more stored demonstrations for augmenting training of the policy for the task in response to a determination that the user input demonstration belongs to the same task as the selected one of the stored demonstrations.

### Claim 19 (depends on 17)

The method of claim 17 wherein the producing the output further includes producing an output that indicates that the one or more stored demonstrations do not correspond to the user input demonstration when their similarity value is less than the predetermined value.

### Claim 20 (depends on 16)

The method of claim 16 further comprising training the attention model using the training dataset of training demonstrations and a listwise loss.

### Claim 21 (depends on 19)

The method of claim 19 further comprising:
determining one or more other stored demonstrations associated with the one or more stored demonstrations in response to the determination that the one or more stored demonstrations correspond to the user input demonstration and belongs to the same task; and
actuating the actuators of the robot further based on the one or more other stored demonstrations.

### Claim 22 (depends on 19)

The method of claim 19 further wherein:
the one or more stored demonstrations include a first time series of at least 6 degree of freedom (DoF) poses of an end effector of the robot; and
the user input demonstration includes a second time series of at least 6 DoF poses of an end effector of the robot.

### Claim 23

A method for operating a robot system comprising:
obtaining a trained policy stored in a memory; and
actuating actuators of a robot using the trained policy thereby performing a task defined by one or more user input demonstrations; and
wherein the trained policy includes a trained attention model that is trained using a training dataset of training demonstrations and a listwise loss over concatenated pairs of the training demonstrations, wherein each of the concatenated pairs of training demonstrations are based on a task identifier associated with each training demonstration,
the training demonstrations each including a temporal sequence of observation-action tuples of successive behaviors for tasks, and
wherein the training dataset of demonstrations, which includes the one or more user input demonstrations for the robot, and which is used to train the trained policy for actuating the actuators of the robot, is augmented with one or more stored demonstrations for the robot when the stored demonstrations are determined to correspond to the one or more user input demonstration and belong to the same task when their similarity values are greater than a predetermined value,
the augmenting including masking one or more scaled outputs and generating a concatenation based on the one or more user input demonstrations and the one or more stored demonstrations.

### Claim 24

A robot system comprising:
a first circuit configured to select a stored demonstration for a robot from a database of stored demonstrations for different tasks of the robot;
a second circuit configured to determine, using an attention model, a similarity value reflecting a similarity between:
a user input demonstration for the robot; and
the selected one of the stored demonstrations for the robot;
wherein the determining the similarity value includes determining the similarity value based on masking one or more scaled outputs and generating a concatenation based on the user input demonstration and the selected one of the stored demonstrations;
a third circuit configured to indicate whether the selected one of the stored demonstrations corresponds to the user input demonstration and belongs to the same task based on the similarity value; and
a fourth circuit configured to actuate actuators of the robot based on a policy trained using the selected one of the stored demonstrations output for augmenting training of the policy,
wherein the attention model is trained using a training dataset of training demonstrations and a listwise loss over concatenated pairs of training demonstrations, wherein each of the concatenated pairs of training demonstrations are based on a task identifier associated with each training demonstration,
the training demonstrations each including a temporal sequence of observation-action tuples of successive behaviors for tasks.