Efficient method for information extraction

A method of extracting information from a text document, comprising:

• finding a best path sequence of states in a HMM, wherein said IBMM is trained using a plurality of training documents each having a sequence of tagged states and said HMM states are modeled with non-exponential length distributions so as to allow their probability length distributions to be changed dynamically during information extraction;
• and
• extracting information from said text document based on said best path sequence of states, wherein if a first state's best transition was from itself, its self-transition probability is adjusted to (1−cdf(t+1))/(1−cdf(t)) and all other outgoing transitions from said first state are scaled by (cdf(t+1)−cdf(t))/(1−cdf(t)), and if said first state is transitioned to by another state, its self-transition probability is reset to its original value of (1−cdf(1))/(1−cdf(0)), where cdf is the cumulative probability distribution function for said first state's length distribution, and t is the number of symbols emitted by said first state in said best path.

A computer-readable medium having computer executable instructions for performing a method of extracting information from a text document, said method comprising:

• finding a best path sequence of states in a HMM, wherein said HMM is trained using a plurality of training documents each having a sequence of tagged states and said HMM states are modeled with non-exponential length distributions so as to allow their probability length distributions to be changed dynamically during information extraction;
• and
• extracting information from said text document based on said best path sequence of states, wherein if a first HMM state's best transition was from itself, its self-transition probability is adjusted to (1−cdf(t+1))/(1−cdf(t)) and all other outgoing transitions from said first HMM state are scaled by (cdf(t+1)−cdf(t))/(1−cdf(t)), and if said first HMM state is transitioned to by another state, its self-transition probability is reset to its original value of (1−cdf(1))/(1−cdf(0)), where cdf is the cumulative probability distribution function for said first state's length distribution, and t is the number of symbols emitted by said first state in said best path.

A method of extracting information from a text document, comprising:

• creating a HMM using a plurality of training documents of a known type, wherein said training documents comprise tagged sequences of states;
• generalizing said HMM by merging repeating sequences of states;
• finding a best path through said HMM representative of said text document, wherein information is extracted from said text document based on said best path.

A method of extracting information from a text document, comprising:

• creating a HMM using a plurality of training documents of a known type, wherein said training documents comprise tagged sequences of states and said HMM comprises HMM states that are modeled with non-exponential length distributions so as to allow their probability length distributions to be changed dynamically during information extraction;
• finding a best path through said HMM representative of said text document, wherein information is extracted from said text document based on said best path, and wherein if a first HMM state's best transition was from itself, its self-transition probability is adjusted to (1−cdf(t+1))/(1−cdf(t)) and all other outgoing transitions from said first HMM state are scaled by (cdf(t+1)−cdf(t))/(1−cdf(t)), and if said first HMM state is transitioned to by another state, its self-transition probability is reset to its original value of (1−cdf(1))/(1−cdf(0)), where cdf is the cumulative probability distribution function for said first state's length distribution, and t is the number of symbols emitted by said first state in said best path.

A computer-readable medium having computer executable instructions for performing a method of extracting information from a text document, said method comprising:

• creating a HMM using a plurality of training documents of a known type, wherein said training documents comprise tagged sequences of states;
• generalizing said HMM by merging repeating sequences of states;
• finding a best path through said HMM representative of said text document, wherein information is extracted from said text document based on said best path.

A computer-readable medium having computer executable instructions for performing a method of extracting information from a text document, said method comprising:

• creating a HMM using a plurality of training documents of a known type, wherein said training documents comprise tagged sequences of states and said HMM comprises HMM states that are modeled with non-exponential length distributions so as to allow their probability length distributions to be changed dynamically during information extraction;
• finding a best path through said HMM representative of said text document, wherein information is extracted from said text document based on said best path, and wherein if a first HMM state's best transition was from itself, its self-transition probability is adjusted to (1−cdf(t+1))/(1−cdf(t)) and all other outgoing transitions from said first HMM state are scaled by (cdf(t+1)−cdf(t))/(1−cdf(t)), and if said first HMM state is transitioned to by another state, its self-transition probability is reset to its original value of (1−cdf(1))/(1−cdf(0)), where cdf is the cumulative probability distribution function for said first state's length distribution, and t is the number of symbols emitted by said first state in said best path.

A computer readable storage medium encoded with information comprising a HMM data structure including a plurality of states in which at least one sequence of states in said HMM data structure is created by merging a repeated sequence of states.

A computer readable storage medium encoded with information comprising a HMM data structure including a plurality of states in which at least one sequence of more than two states in said HMM data structure includes a transition from a last state in the at least one sequence to the first state in the sequence.