Generating a model for an object encountered by a robot
AssigneeX Development LLC
InventorsKurt Konolige, Nareshkumar Rajkumar, Stefan Hinterstoisser
Life of this patent
Estimating the term…
Abstract
Methods and apparatus related to generating a model for an object encountered by a robot in its environment, where the object is one that the robot is unable to recognize utilizing existing models associated with the robot. The model is generated based on vision sensor data that captures the object from multiple vantages and that is captured by a vision sensor associated with the robot, such as a vision sensor coupled to the robot. The model may be provided for use by the robot in detecting the object and/or for use in estimating the pose of the object.
Claims15
A method, comprising:
- identifying vision sensor data generated by a vision sensor coupled to a robot, the vision sensor data capturing a portion of an environment of the robot, including an object in the environment;
- determining, based on output generated from application of the vision sensor data to a machine learning model, that the machine learning model fails to detect the object with at least a threshold level of confidence;
- in response to determining the machine learning model fails to detect the object:
- generating a plurality of training examples based on one or both of:
- the vision sensor data, and additional vision sensor data that is generated by the vision sensor and that captures the object, and
- training the machine learning model based on the plurality of training examples;
- and
- after training the machine learning model based on the plurality of training examples:
- using, by the robot, the machine learning model that has been trained based on the plurality of training examples to detect the object based on further vision sensor data generated by the vision sensor coupled to the robot.
A system, comprising:
- a robot comprising one or more robot processors and a vision sensor capturing a portion of an environment of the robot, wherein the portion of the environment includes an object, and wherein the one or more robot processors are configured to:
- identify vision sensor data generated by the vision sensor, and
- determine, based on output generated from application of the vision sensor data to a machine learning model, that the machine learning model fails to detect the object with at least a threshold level of confidence;
- a model generation system implemented by one or more processors, wherein the model generation system is configured to:
- in response to determining that the machine learning model fails to detect the object:
- generate a plurality of training examples based on one or both of:
- the vision sensor data, and additional vision sensor data that is generated by the vision sensor and that captures the object, and
- train the machine learning model based on the plurality of training examples;
- wherein after training the machine learning model based on the plurality of training examples, one or more of the robot processors use the machine learning model that has been trained based on the plurality of training examples to detect the object based on further vision sensor data generated by the vision sensor coupled to the robot.
Dependencies are inferred from the claim text.