Back to results
US 12,596,955 B2Granted
Reward feedback for learning control policies using natural language and vision data
AssigneeHitachi, Ltd.
InventorsAndrew James Walker, Joydeep Acharya
Life of this patent
Estimating the term…
Abstract
Example implementations described herein involve systems and methods for providing a reward to a machine learning algorithm, which can include receiving an image, and a task description defined in text; slicing the image into a plurality of sub-images; executing an embedding model to embed the text of the task description and the sub-images to generate a distribution for the sub-images based on relevance to the task description; and generating the reward from the distribution for the sub-images.
Claims9
A method for providing a reward to a machine learning algorithm, comprising:
- receiving an image, and a task description defined in text;
- slicing the image into a plurality of sub-images;
- executing an embedding model to embed the text of the task description and the sub-images to generate a distribution for the sub-images based on relevance to the task description;
- generating the reward from the distribution for the sub-images;
- and
- in response to receiving a selection of text for the task description, update the task description based on the selection of text by performing steps comprising:
- extracting, from a database of images, a set of images associated with successful episodes resulting from output of the machine learning algorithm:
- extracting, from the set of images, another set of images that correspond to goals related to the selection of text;
- executing a semantic extractor on the another set of images and the selection of text to produce an embedding of the another set of images and the selection of the text in a semantic space;
- determining a closest point in the semantic space to the another set of images and the selection of the text;
- and
- generating an updated task description from the closest point in the semantic space, the updated task description comprising a vector in the semantic space.
9 more elements
A non-transitory computer readable medium, storing instructions for providing a reward to a machine learning algorithm, the instructions comprising:
- receiving an image, and a task description defined in text;
- slicing the image into a plurality of sub-images;
- executing an embedding model to embed the text of the task description and the sub-images to generate a distribution for the sub-images based on relevance to the task description;
- generating the reward from the distribution for the sub-images;
- and
- in response to receiving a selection of text for the task description, update the task description based on the selection of text by performing steps comprising:
- extracting, from a database of images, a set of images associated with successful episodes resulting from output of the machine learning algorithm;
- extracting, from the set of images, another set of images that correspond to goals related to the selection of text;
- executing a semantic extractor on the another set of images and the selection of text to produce an embedding of the another set of images and the selection of the text in a semantic space:
- determining a closest point in the semantic space to the another set of images and the selection of the text;
- and
- generating an updated task description from the closest point in the semantic space, the updated task description comprising a vector in the semantic space.
9 more elements
A system, comprising:
- one or more robotic control systems, the one or more robotic control systems configured to control one or more robots;
- and
- an apparatus, comprising:
- a processor configured to provide a reward to a machine learning algorithm controlling the one or more robotic control systems, the processor configured to:
- receive an image, and a task description defined in text;
- slice the image into a plurality of sub-images;
- execute an embedding model to embed the text of the task description and the sub- images to generate a distribution for the sub-images based on relevance to the task description:
- generate the reward from the distribution for the sub-images;
- and
- in response to receiving a selection of text for the task description, update the task description based on the selection of text by performing steps comprising:
- extracting, from a database of images, a set of images associated with successful episodes resulting from output of the machine learning algorithm;
- extracting, from the set of images, another set of images that correspond to goals related to the selection of text:
- executing a semantic extractor on the another set of images and the selection of text to produce an embedding of the another set of images and the selection of the text in a semantic space:
- determining a closest point in the semantic space to the another set of images and the selection of the text;
- and
- generating an updated task description from the closest point in the semantic space, the updated task description comprising a vector in the semantic space.
13 more elements
Dependencies are inferred from the claim text.