Autonomous task performance based on visual embeddings
AssigneeToyota Research Institute, Inc.
InventorsJeremy Ma, Kevin Stone, Max Bajracharya, Krishna Shankar
Life of this patent
Estimating the term…
Abstract
A method for controlling a robotic device is presented. The method includes capturing an image corresponding to a current view of the robotic device. The method also includes identifying a keyframe image comprising a first set of pixels matching a second set of pixels of the image. The method further includes performing, by the robotic device, a task corresponding to the keyframe image.
Claims19
A method for controlling a robotic device, comprising:
- capturing an image corresponding to a current view of the robotic device;
- identifying a keyframe image based on matching one or more pixels of a first set of pixels associated with the keyframe image with one or more pixels of a second set of pixels associated with the image;
- determining, based on identifying the keyframe, a difference between a first relationship of a keyframe robotic device to a first object in the keyframe image and a second relationship of the robotic device to a second object in the image;
- adjusting a parameter associated with the task based on determining the difference between the first relationship and the second relationship, the task being associated with the keyframe image;
- and
- controlling the robotic device to perform a task based on adjusting the parameter.
An apparatus for controlling a robotic device, comprising:
- a processor;
- and
- a memory coupled with the processor and storing instructions operable, when executed by the processor, to cause the apparatus to:
- capture an image corresponding to a current view of the robotic device;
- identify a keyframe image based on matching one or more pixels of a first set of pixels associated with the keyframe image with one or more pixels of a second set of pixels associated with the image;
- determine, based on identifying the keyframe, a difference between a first relationship of a keyframe robotic device to a first object in the keyframe image and a second relationship of the robotic device to a second object in the image;
- adjust a parameter associated with the task based on determining the difference between the first relationship and the second relationship, the task being associated with the keyframe image;
- and
- control the robotic device to perform a task based on adjusting the parameter.
A non-transitory computer-readable medium having program code recorded thereon for controlling a robotic device, the program code comprising:
- program code to capture an image corresponding to a current view of the robotic device;
- program code to identify a keyframe image based on matching one or more pixels of a first set of pixels associated with the keyframe image with one or more pixels of a second set of pixels associated with the image;
- program code to determine, based on identifying the keyframe, a difference between a first relationship of a keyframe robotic device to a first object in the keyframe image and a second relationship of the robotic device to a second object in the image;
- program code to adjust a parameter associated with the task based on determining the difference between the first relationship and the second relationship, the task being associated with the keyframe image;
- and
- program code to control the robotic device to perform a task based on adjusting the parameter.
Dependencies are inferred from the claim text.