Back to results
US 10,796,098 B2Granted
Instruction understanding system and instruction understanding method
AssigneeNational Institute of Information and Communications Technology
InventorsKomei Sugiura, Hisashi Kawai
Life of this patent
Estimating the term…
Abstract
A new technology of prediction of manipulability in response even to an instruction with missing information in an object manipulation task to have a robot manipulate some kind of object is provided. An instruction understanding system includes an obtaining engine configured to obtain a linguistic expression of a name of an object to be manipulated and a linguistic expression of a situation where the object corresponding to the name is placed in a real environment and a classifier configured to receive input of the linguistic expression of the name and the linguistic expression of the situation and output manipulability of the object corresponding to the name in the real environment.
Claims17
An instruction understanding system for language understanding of an instruction sentence comprising:
- at least one of (a) a processor executing a computer program and (b) hard-wired circuit(s), programmed to operate as:
- an obtaining engine configured to obtain a linguistic expression of a name of an object to be manipulated and a linguistic expression of a situation where the object corresponding to the name is placed in a real environment;
- and
- a classifier configured to receive input of the linguistic expression of the name and the linguistic expression of the situation and output manipulability of the object corresponding to the name in the real environment, the manipulability of the object expressed as a numerical value or category representing a likelihood that the object can be manipulated in some way,
- wherein the classifier has been trained using a set of a linguistic expression of a name of an object in an image obtained from each piece of image data included in an image data set, a linguistic expression of a situation about the object, and a label of manipulability, and
- wherein the classifier includes:
- an extraction network configured to generate a feature sample upon receiving input of the linguistic expression of the name and the linguistic expression of the situation, and
- a discrimination network configured to output, upon receiving input of the feature sample, a first predicted value representing a category of manipulability to which the feature sample is most likely to belong and a second predicted value representing a likelihood of generation of the feature sample by the extraction network,
- wherein the discrimination network is configured to receive from a generation network configured to generate a pseudo sample upon receiving input of the category and noise, input of the pseudo sample, and
- wherein the discrimination network has been trained using an object function including a cost function about an error in discrimination as to whether an input sample is the feature sample or the pseudo sample and a cost function about an error in discrimination of the category to which the input sample belongs.
7 more elements
An instruction understanding method for language understanding of an instruction sentence, said method being practiced by at least one of (a) a processor executing a computer program and (b) hard-wired circuit(s), programmed for executing the steps of:
- obtaining a linguistic expression of a name of an object to be manipulated and a linguistic expression of a situation where the object corresponding to the name is placed in a real environment;
- and
- inputting the linguistic expression of the name and the linguistic expression of the situation into a classifier and outputting manipulability of the object corresponding to the name in the real environment, the manipulability of the object expressed as a numerical value or category representing a likelihood that the object can be manipulated in some way,
- wherein the classifier has been trained using a set of a linguistic expression of a name of an object in an image obtained from each piece of image data included in an image data set, a linguistic expression of a situation about the object, and a label of manipulability, and
- wherein the classifier comprises at least one of (a) a processor executing a computer program and (b) hard-wired circuit(s), programmed to operate as:
- an extraction network configured to generate a feature sample upon receiving input of the linguistic expression of the name and the linguistic expression of the situation, and
- a discrimination network configured to output, upon receiving input of the feature sample, a first predicted value representing a category of manipulability to which the feature sample is most likely to belong and a second predicted value representing a likelihood of generation of the feature sample by the extraction network,
- wherein the discrimination network is configured to receive from a generation network configured to generate a pseudo sample upon receiving input of the category and noise, input of the pseudo sample, and
- wherein the discrimination network has been trained using an object function including a cost function about an error in discrimination as to whether an input sample is the feature sample or the pseudo sample and a cost function about an error in discrimination of the category to which the input sample belongs.
6 more elements
An instruction understanding system for language understanding of an instruction sentence comprising:
- at least one of (a) a processor executing a computer program and (b) hard-wired circuit(s), programmed to operate as:
- an obtaining engine configured to obtain a linguistic expression of a name of an object to be manipulated and a linguistic expression of a situation where the object corresponding to the name is placed in a real environment;
- and
- a classifier configured to receive input of the linguistic expression of the name and the linguistic expression of the situation and output manipulability of the object corresponding to the name in the real environment, the manipulability of the object expressed as a numerical value or category representing a likelihood that the object can be manipulated in some way,
- wherein the classifier has been trained using a set of a linguistic expression of a name of an object in an image obtained from each piece of image data included in an image data set, a linguistic expression of a situation about the object, and a label of manipulability, and
- wherein the classifier comprises
- an extraction network configured to generate a feature sample upon receiving input of the linguistic expression of the name and the linguistic expression of the situation, and
- a discrimination network configured to output, upon receiving input of the feature sample, a first predicted value representing a category of manipulability to which the feature sample is most likely to belong and a second predicted value representing a likelihood of generation of the feature sample by the extraction network.
5 more elements
Dependencies are inferred from the claim text.