Suggestion of new entity types with discriminative term importance analysis
AssigneeInternational Business Machines Corporation
InventorsHaode Qi, Ming Tan, Yang Yu, Navneet N. Rao, Saloni Potdar, Haoyu Wang
Life of this patent
Estimating the term…
Abstract
A mechanism is provided to implement suggestion of new entity types with discriminative importance analysis. The mechanism obtains a list of predefined intents from a chatbot designer. The mechanism receives an input sentence having a target intent within the list of predefined intents. The mechanism performs intent-specific importance analysis on the input sentence to generate an importance score for each token in the input sentence. The mechanism ranks the tokens in the input sentence by importance score and outputs a token with a highest importance score as a candidate entity type.
Claims18
A method, in a data processing system comprising a processor and a memory, the memory comprising instructions that are executed by the processor to specifically configure the processor to implement suggestion of new entity types with discriminative importance analysis, the method comprising:
- receiving, in an artificial intelligence conversational computing system, an input natural language sentence;
- processing the input natural language sentence to identify a target intent of the input natural language sentence;
- generating a set of adversarial examples based on the input natural language sentence, each adversarial example having a corresponding adversarial example intent to thereby generate a list of predefined intents;
- performing intent-specific importance analysis on the input sentence to generate an importance score for each token in the input sentence at least by:
- generating an importance score for each token of the input sentence based on sensitivity to the target intent;
- adjusting the importance score for one or more tokens in the input sentence based on sensitivity of the one or more tokens to one or more of the adversarial example intents in the list of predefined intents;
- and
- ranking the tokens in the input sentence based on their corresponding importance scores to generate a ranked set of tokens;
- and
- configuring the artificial intelligence conversational computing system to process a token with a highest rank, in the ranked set of tokens, as a new entity in conversational recognition during an automated dialog with a user.
A computer program product comprising a computer readable storage medium having a computer readable program stored therein, wherein the computer readable program, when executed on a processor of a data processing system, causes the data processing system to implement suggestion of new entity types with discriminative importance analysis, wherein the computer readable program causes the data processing system to:
- receive, in an artificial intelligence conversational computing system, an input natural language sentence;
- process the input natural language sentence to identify a target intent of the input natural language sentence;
- generate a set of adversarial examples based on the input natural language sentence, each adversarial example having a corresponding adversarial example intent to thereby generate a list of predefined intents;
- perform intent-specific importance analysis on the input sentence to generate an importance score for each token in the input sentence at least by:
- generating an importance score for each token of the input sentence based on sensitivity to the target intent;
- adjusting the importance score for one or more tokens in the input sentence based on sensitivity of the one or more tokens to one or more of the adversarial example intents in the list of predefined intents;
- and
- ranking the tokens in the input sentence based on their corresponding importance scores to generate a ranked set of tokens;
- and
- output configure the artificial intelligence conversational computing system to process a token with a highest rank, in the ranked set of tokens, as a new entity in conversational recognition during an automated dialog with a user.
An apparatus comprising:
- a processor;
- and
- a memory coupled to the processor, wherein the memory comprises instructions which, when executed by the processor, cause the processor to implement suggestion of new entity types with discriminative importance analysis, wherein the instructions cause the processor to:
- receive, in an artificial intelligence conversational computing system, an input natural language sentence;
- process the input natural language sentence to identify a target intent of the input natural language sentence;
- generate a set of adversarial examples based on the input natural language sentence, each adversarial example having a corresponding adversarial example intent to thereby generate a list of predefined intents;
- perform intent-specific importance analysis on the input sentence to generate an importance score for each token in the input sentence at least by:
- generating an importance score for each token of the input sentence based on sensitivity to the target intent;
- adjusting the importance score for one or more tokens in the input sentence based on sensitivity of the one or more tokens to one or more of the adversarial example intents in the list of predefined intents;
- and
- ranking the tokens in the input sentence based on their corresponding importance scores to generate a ranked set of tokens;
- and
- output configure the artificial intelligence conversational computing system to process a token with a highest rank, in the ranked set of tokens, as a new entity in conversational recognition during an automated dialog with a user.
Dependencies are inferred from the claim text.