Back to results
US 12,304,081 B2Granted
Deep compositional robotic planners that follow natural language commands
AssigneeMassachusetts Institute of Technology
InventorsYen-Ling Kuo, Boris Katz, Andrei Barbu
Life of this patent
Estimating the term…
Abstract
The present approach similarly combines task and motion planning, but does so without symbolic representations and begins with simpler tasks than other models in such domains can handle. Unlike prior approaches, the present approach does so in continuous action and state spaces which require many precise steps in the configuration space to execute what otherwise is a single output token such as “pick up” for discrete problems.
Claims13
A method comprising:
- determining, by a planner, a first point in a selected neighborhood of a configuration space;
- receiving a natural language input, uttered by a human or generated by a computer;
- parsing the received natural language input to determine (i) linguistic structure of the natural language input and (ii) a plurality of words in the natural language input;
- modifying a first neural network (NN) to encode the determined linguistic structure of the natural language input by arranging a plurality of component neural networks of the first NN in an order corresponding to the determined linguistic structure of the natural language input, wherein each component neural network corresponds to one or more respective words of the determined plurality of words;
- determining, by the modified first NN, a second point in the selected neighborhood of the configuration space;
- choosing among the first point and second point to generate an additional node to add to a search tree;
- and
- adding the additional node to the search tree by connecting the additional node to a node associated with the selected neighborhood;
- wherein a second NN controls behavior of an agent corresponding to the search tree having the additional node.
6 more elements
A system comprising:
- a processor;
- and
- a memory with computer code instructions stored thereon, the processor and the memory, with the computer code instructions, being configured to cause the processor to:
- determine, by a planner, a first point in a selected neighborhood of a configuration space;
- receive a natural language input, uttered by a human or generated by a computer;
- parse the received natural language input to determine (i) linguistic structure of the natural language input and (ii) a plurality of words in the natural language input;
- modify a first neural network (NN) to encode the determined linguistic structure of the natural language input by arranging a plurality of component neural networks of the first NN in an order corresponding to the determined linguistic structure of the natural language input, wherein each component neural network corresponds to one or more respective words of the determined plurality of words,
- determine, by the modified first NN, a second point in the selected neighborhood of the configuration space;
- choose among the first point and second point to generate an additional node to add to a search tree;
- and
- add the additional node to the search tree by connecting the additional node to a node associated with the selected neighborhood;
- wherein a second NN controls behavior of an agent corresponding to the search tree having the additional node.
9 more elements
A method comprising:
- determining, by a planner, a first neighborhood of a configuration space and a first point in the first neighborhood;
- receiving a natural language input, uttered by a human or generated by a computer;
- parsing the received natural language input to determine (i) linguistic structure of the natural language input and (ii) a plurality of words in the natural language input;
- modifying a first neural network (NN) to encode the determined linguistic structure of the natural language input by arranging a plurality of component neural networks of the first NN in an order corresponding to the determined linguistic structure of the natural language input, wherein each component neural network corresponds to one or more respective words of the determined plurality of words;
- determining, by the modified first NN, a second neighborhood of a configuration space and a second point in the second neighborhood;
- choosing as a selected neighborhood, one neighborhood among the first neighborhood and second neighborhood based on at least one of:
- a respective level of confidence determined for the first neighborhood and second neighborhood, at least one extrinsic factor, and an impossibility factor indicative of whether a path to each node of the selected neighborhood is possible;
- choosing among the first point and second point to generate an additional node to add to a search tree;
- and
- adding the additional node to the search tree by connecting the additional node to a node associated with the selected neighborhood;
- wherein a second NN controls behavior of an agent corresponding to the search tree having the additional node.
8 more elements
Dependencies are inferred from the claim text.