Method and apparatus for automatic identification of word boundaries in continuous text and computation of word boundary scores
AssigneeLernout & Hauspie Speech Products N.V.
InventorsAlwin B. Carus, Kathleen Good
Life of this patent
Estimating the term…
Abstract
A method and device for identifying word boundaries in continuous text compares the continuous text to a set of varying length strings to identify candidate word-initial boundaries and candidate word-final boundaries in the continuous text. Each candidate word-initial boundary and candidate word-final boundary has an associated probability value. Each candidate word boundary in the continuous text is identified by calculating a word boundary score for such candidate word boundary using the probability values associated with the candidate word-initial boundaries and candidate word-final boundaries. The set of varying length strings may include words and n-grams.
Claims20
A computerized method for identifying word boundaries in a continuous text input, the method comprising the following digital processes:
- (a) comparing the continuous text to a set of varying length strings to identify candidate word-initial boundaries and candidate word-final boundaries in the continuous text, each candidate word-initial boundary and candidate word-final boundary being a character in the continuous text and having an associated probability value;
- (b) identifying each candidate word boundary in the continuous text by calculating a word boundary score for such candidate word boundary using the probability values associated with the candidate word-initial boundaries and candidate word-final boundaries identified in step (a), the candidate word boundaries defining segments of the continuous text;
- and
- (c) verifying each segment defined by the candidate word boundaries identified in step (b) against a string database.
A computerized data processing device for identifying word boundaries in a continuous text input, the device comprising:
- a string comparator, to identify candidate word-initial boundaries and candidate word-final boundaries in the continuous text by comparing the continuous text to a set of varying length strings, each candidate word-initial boundary and candidate word-final boundary being a character in the continuous text and having an associated probability value;
- a boundary checker, coupled to the string comparator, to identify each candidate word boundary in the continuous text by calculating a word boundary score for such candidate word boundary using the probability values associated with the candidate word-initial boundaries and candidate word-final boundaries identified by the string comparator, the candidate word boundaries defining segments of the continuous text.
A digital storage medium encoded with instructions which, when loaded into a computer, establishes a device for identifying word boundaries in continuous text, the device including:
- a string comparator, to identify candidate word-initial boundaries and candidate word-final boundaries in the continuous text by comparing the continuous text to a set of varying length strings, each candidate word-initial boundary and candidate word-final boundary being a character in the continuous text and having an associated probability value;
- and
- a boundary checker, coupled to the string comparator, to identify each candidate word boundary in the continuous text by calculating a word boundary score for such candidate word boundary using the probability values associated with the candidate word-initial boundaries and candidate word-final boundaries identified by the string comparator, the candidate word boundaries defining segments of the continuous text.
Dependencies are inferred from the claim text.