Back to results
US 11,288,883 B2Granted
Autonomous task performance based on visual embeddings
AssigneeToyota Research Institute, Inc.
InventorsJeremy Ma, Kevin Stone, Max Bajracharya, Krishna Shankar
Life of this patent
Estimating the term…
Abstract
A method for controlling a robotic device is presented. The method includes capturing an image corresponding to a current view of the robotic device. The method also includes identifying a keyframe image comprising a first set of pixels matching a second set of pixels of the image. The method further includes performing, by the robotic device, a task corresponding to the keyframe image.
Claims20
A method for controlling a robotic device, comprising:
- capturing an image corresponding to a current view of the robotic device;
- identifying a keyframe image based on a red-green-blue (RGB) value of each pixel of a first set of pixels of the keyframe image matching an RGB value of a corresponding pixel of a second set of pixels of the image;
- determining one or both of:
- a first difference between a first pose of the robotic device in relation to a first object in the keyframe image and a second pose of the robotic device in relation to a second object in the image;
- or
- a second difference between a first distance of the robotic device to the first object and a second distance of the robotic device to the second object;
- adjusting one or more of a velocity or a position associated with a task to be performed by an effecter of the robotic device based on determining one or both of the first difference or the second difference, the task being associated with the keyframe image;
- and
- performing, via the effecter of the robotic device, the task based on adjusting one or both of the velocity or the position.
6 more elements
A robotic device, comprising:
- a memory;
- and
- at least one processor, the at least one processor configured:
- to capture an image corresponding to a current view of the robotic device;
- to identify a keyframe image based on a red-green-blue (RGB) value of each pixel of a first set of pixels of the keyframe image matching an RGB value of a corresponding pixel of a second set of pixels of the image;
- to determine one or both of:
- a first difference between a first pose of the robotic device in relation to a first object in the keyframe image and a second pose of the robotic device in relation to a second object in the image;
- or
- a second difference between a first distance of the robotic device to the first object and a second distance of the robotic device to the second object;
- to adjust one or more of a velocity or a position associated with a task to be performed by an effecter of the robotic device based on determining one or both of the first difference or the second difference, the task being associated with the keyframe image;
- and
- to perform, via the effecter of the robotic device, the task based on adjusting one or both of the velocity or the position.
9 more elements
A non-transitory computer-readable medium having program code recorded thereon for controlling a robotic device, the program code comprising:
- program code to capture an image corresponding to a current view of the robotic device;
- program code to identify a keyframe image based on a red-green-blue (RGB) value of each pixel of a first set of pixels of the keyframe image matching an RGB value of a corresponding pixel of a second set of pixels of the image;
- program code to determine one or both of:
- a first difference between a first pose of the robotic device in relation to a first object in the keyframe image and a second pose of the robotic device in relation to a second object in the image;
- or
- a second difference between a first distance of the robotic device to the first object and a second distance of the robotic device to the second object;
- program code to adjust one or more of a velocity or a position associated with a task to be performed by an effecter of the robotic device based on determining one or both of the first difference or the second difference, the task being associated with the keyframe image;
- and
- program code to perform, via the effecter of the robotic device, the task based on adjusting one or both of the velocity or the position.
6 more elements
Dependencies are inferred from the claim text.