# US 12,762,848 B2 · Device and method for training a control policy for manipulating an object

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,762,848 B2
- Document ID: US12762848B2
- Status: Granted
- Kind: B2
- Application number: 18/931,409
- Filed: 2024-10-30
- Published: 2025-06-19
- Granted: 2026-10-13
- Earliest claimed: 2023-12-15
- Assignees: Robert Bosch GmbH
- Applicants: Robert Bosch GmbH
- Inventors: Alexander Qualmann; Anh Vien Ngo; Bao Huy Le; Miroslav Gabriel; Philipp Christian Schillinger
- Web page: https://patentgrep.com/patent/US12762848B2

## CPC

- B25J 9/1697: {Vision controlled systems}
- G06T 7/50: Depth or shape recovery
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06T 11/60: Creating or editing images; Combining images with text
- G06V 20/70: Labelling scene content, e.g. deriving syntactic or semantic representations
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50})
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A method for training a control policy for manipulating an object. For each of one or more objects in each of one or more scenes, receiving an input data element including image data representing a shape of the object to be manipulated and its position in the scene, generating, for each input data element, one or more training data elements by generating augmentations of the image data and pseudo-labels for the augmented image data according to a semi-supervised learning scheme and training the control policy using the generated training data elements.

## Claims

### Claim 1

A method for training a control policy for manipulating an object, comprising the following steps:
for each of one or more objects in each of one or more scenes:
receiving an input data element including image data representing a shape of the object to be manipulated and a position of the object in the scene;
generating, for each input data element, one or more training data elements by generating augmentations of the image data and pseudo-labels for the augmented image data according to a semi-supervised learning scheme; and
training the control policy using the generated training data elements;
wherein the training of the control policy includes determining a loss including loss terms for the generated training data elements, wherein: (i) each loss term is soft-weighted in a loss function by applying a softmax function to a confidence of pseudo-labels of manipulation poses of a respective training data element of the generated training data elements and/or (ii) loss terms are filtered out of the loss function when the confidence of pseudo-labels of manipulation poses of respective training data elements of the generated training data elements is below a predetermined threshold.

### Claim 2 (depends on 1)

The method of claim 1, further comprising training the control policy using reinforcement learning.

### Claim 3 (depends on 2)

The method of claim 2, further comprising training the control policy using actor-critic reinforcement learning and wherein training the control policy includes training an actor and a critic using the generated training data elements.

### Claim 4 (depends on 1)

The method of claim 1, wherein the training of the control policy includes training a neural network representing the control policy.

### Claim 5 (depends on 1)

The method of claim 1, wherein, for each generated training data element, a pseudo label is generated for each of a plurality of manipulation poses, wherein each manipulation pose includes a manipulation position corresponding to a respective pixel in the respective augmented image data.

### Claim 6 (depends on 1)

The method of claim 1, wherein the threshold is a pixel-wise threshold.

### Claim 7

A method for controlling a robot device, comprising the following steps:
training a control policy for manipulating an object by:
for each of one or more objects in each of one or more scenes:
receiving an input data element including image data representing a shape of the object to be manipulated and a position of the object in the scene,
generating, for each input data element, one or more training data elements by generating augmentations of the image data and pseudo-labels for the augmented image data according to a semi-supervised learning scheme, and
training the control policy using the generated training data elements,
wherein the training of the control policy includes determining a loss including loss terms for the generated training data elements, wherein: (i) each loss term is soft-weighted in a loss function by applying a softmax function to a confidence of pseudo-labels of manipulation poses of a respective training data element of the generated training data elements and/or (ii) loss terms are filtered out of the loss function when the confidence of pseudo-labels of manipulation poses of respective training data elements of the generated training data elements is below a predetermined threshold;
receiving, for a scene in which the robot device should be controlled, further image data representing the scene; and
supplying the obtained further image data to the control policy and generating a control signal for the robot device according to an output that the control policy generates in response to the obtained further image data.

### Claim 8

A data processing device, configured to train a control policy for manipulating an object, the data processing device configured to:
for each of one or more objects in each of one or more scenes:
receive an input data element including image data representing a shape of the object to be manipulated and a position of the object in the scene;
generate, for each input data element, one or more training data elements by generating augmentations of the image data and pseudo-labels for the augmented image data according to a semi-supervised learning scheme; and
train the control policy using the generated training data elements;
wherein the training of the control policy includes determining a loss including loss terms for the generated training data elements, wherein: (i) each loss term is soft-weighted in a loss function by applying a softmax function to a confidence of pseudo-labels of manipulation poses of a respective training data element of the generated training data elements and/or (ii) loss terms are filtered out of the loss function when the confidence of pseudo-labels of manipulation poses of respective training data elements of the generated training data elements is below a predetermined threshold.

### Claim 9

A non-transitory computer-readable medium on which is stored a computer program including instructions for training a control policy for manipulating an object, the instructions, when executed by a computer, causing the computer to perform the following steps:
for each of one or more objects in each of one or more scenes:
receiving an input data element including image data representing a shape of the object to be manipulated and a position of the object in the scene;
generating, for each input data element, one or more training data elements by generating augmentations of the image data and pseudo-labels for the augmented image data according to a semi-supervised learning scheme; and
training the control policy using the generated training data elements;
wherein the training of the control policy includes determining a loss including loss terms for the generated training data elements, wherein: (i) each loss term is soft-weighted in a loss function by applying a softmax function to a confidence of pseudo-labels of manipulation poses of a respective training data element of the generated training data elements, and/or (ii) loss terms are filtered out of the loss function when the confidence of pseudo-labels of manipulation poses of respective training data elements of the generated training data elements is below a predetermined threshold.