Name entity extraction using language models

A method, in a data processing system, for name entity extraction, comprising:

• a) providing a general language model;
• b) setting initial probabilities in the general language model;
• c) extracting name entities in a plurality of training sentences;
• d) replacing the extracted name entities with name entity labels in the plurality of training sentences to form modified training sentences;
• e) training the general language model using the modified training sentences;
• and
• f) repeating steps (c) to (e) until probabilities the general language model do not change.

An apparatus for name entity extraction, comprising:

• means for providing a general language model and one or more name entity language models, wherein the general language model and the one or more name entity language models are associated with a plurality of states including a general language model state and one or more name entity language model states associated with the one or more name entity language models;
• means for receiving a natural language utterance;
• means, starting at a start node in the general language model, for each word in the natural language utterance, for determining a best current state in the plurality of states and a best previous state in the plurality of states, until an end node in the general language model is reached;
• means for tracing back to determine a best path from the start node to the end node;
• means for identifying two or more adjacent words in the natural language utterance aligned with a name entity language model state as a name entity;
• and
• means for extracting the name entity.

An apparatus for name entity extraction, comprising:

• a processor;
• and
• a memory electrically connected to the processor, the memory having stored therein a program to be executed on the processor for performing the following steps:
• a) providing a general language model;
• b) setting initial probabilities in the general language model;
• c) extracting name entities in a plurality of training sentences;
• d) replacing the extracted name entities with name entity labels in the plurality of training sentences to form modified training sentences;
• e) training the general language model using the modified training sentences;
• and
• f) repeating steps (c) to (e) until probabilities the general language model do not change.

A computer program product, in a computer readable medium, for name entity extraction, comprising:

• instructions for providing a general language model and one or more name entity language models, wherein the general language model and the one or more name entity language models are associated with a plurality of states including a general language model state and one or more name entity language model states associated with the one or more name entity language models;
• instructions for receiving a natural language utterance;
• instructions for, starting at a start node in the general language model, for each word in the natural language utterance, determining a best current state in the plurality of states and a best previous state in the plurality of states, until an end node in the general language model is reached;
• instructions for tracing back to determine a best path from the start node to the end node;
• instructions for identifying two or more adjacent words in the natural language utterance aligned with a name entity language model state as a name entity;
• and
• instructions for extracting the name entity.

A computer program product, in a computer readable medium, for name entity extraction, comprising:

• a) instructions for providing a general language model;
• b) instructions for setting initial probabilities in the general language model;
• c) instructions for extracting name entities in a plurality of training sentences;
• d) instructions for replacing the extracted name entities with name entity labels in the plurality of training sentences to form modified training sentences;
• and
• e) instructions for training the general language model using the modified training sentences,
• wherein instructions (c) to (e) are repeated until probabilities the general language model do not change.