Device and method for controlling a robot

A robot controller, configured to control a robot, the robot controller configured to:

• Perform demonstrations, wherein each demonstration of the demonstrations demonstrates a handling of an object;
• Provide, for each demonstration, at least one descriptor image of the object from a point of view of the robot, wherein the descriptor image specifies feature points for locations on the object;
• Select a set of feature points from the specified feature points, wherein the feature points are selected by searching an optimum of an objective function which rewards selected feature points being visible in the descriptor images;
• Train a robot control model using the demonstrations, wherein the robot control model is configured to output control information depending on an input object pose;
• and
• Control the robot for a control scene with the object by:
• Determining a descriptor image of the object from the point of view of the robot;
• Locating the selected set of feature points in the descriptor image of the object;
• Determining Euclidean coordinates of the located feature points for the control scene;
• Estimating a pose from the determined Euclidean coordinates;
• and
• Controlling the robot to handle the object using the robot control model, wherein the estimated pose is supplied to the robot control model as input.

A non-transitory computer-readable medium on which is stored a computer program for controlling a robot, the control program, when executed by a computer, causing the computer to perform:

• Performing demonstrations, wherein each demonstration of the demonstrations demonstrates a handling of an object;
• Providing, for each demonstration, at least one descriptor image of the object from a point of view of the robot, wherein the descriptor image specifies feature points for locations on the object;
• Selecting a set of feature points from the specified feature points, wherein the feature points are selected by searching an optimum of an objective function which rewards selected feature points being visible in the descriptor images;
• Training a robot control modelusing the demonstrations, wherein the robot control model is configured to output control information depending on an input object pose;
• and
• Controlling the robot for a control scene with the object by:
• Determining a descriptor image of the object from the point of view of the robot;
• Locating the selected set of feature points in the descriptor image of the object;
• Determining Euclidean coordinates of the located feature points for the control scene;
• Estimating a pose from the determined Euclidean coordinates;
• and
• Controlling the robot to handle the object using the robot control model, wherein the estimated pose is supplied to the robot control model as input.