Bi-dimensional rewriting rules for natural language processing
AssigneeXerox Corporation
InventorsCaroline Brun, Caroline Hagege, Claude Roux
Life of this patent
Estimating the term…
Abstract
A linguistic rewriting rule for use in linguistic processing of an ordered sequence of linguistic tokens includes a token pattern recognition rule that matches the ordered sequence of linguistic tokens with a syntactical pattern. The token pattern recognition rule incorporates a character pattern recognition rule to match characters contained in an ambiguous portion of the ordered sequence of linguistic tokens with a character pattern defining a corresponding portion of the syntactical pattern.
Claims23
a storage medium storing instructions which when executed by an associated digital processor implement a rewriting rule for use in linguistic processing of an ordered sequence of linguistic tokens, the rewriting rule including (i) a character pattern recognition rule and (ii) a token pattern recognition rule matching the ordered sequence of linguistic tokens with a syntactical pattern, the token pattern recognition rule incorporating the character pattern recognition rule to match characters contained in an ambiguous portion of the ordered sequence of linguistic tokens with a character pattern defining a corresponding portion of the syntactical pattern.
A linguistic rewriting rule for use in linguistic processing of an ordered sequence of linguistic tokens, the rewriting rule comprising:
- a character pattern recognition rule;
- and
- a token pattern recognition rule matching the ordered sequence of linguistic tokens with a syntactical pattern, the token pattern recognition rule incorporating the character pattern recognition rule to match characters contained in an ambiguous portion of the ordered sequence of linguistic tokens with a character pattern defining a corresponding portion of the syntactical pattern.
A linguistic processing method for processing an ordered sequence of linguistic tokens, the linguistic processing method comprising:
- attempting matching of the ordered sequence of linguistic tokens with a syntactical pattern, wherein at least a portion of the attempted matching is performed by attempting matching of characters contained in an ambiguous portion of the ordered sequence of linguistic tokens with a character pattern;
- and
- categorizing at least one of:
- (i) the ordered sequence of linguistic tokens, (ii) an ordered sub-sequence of the ordered sequence of linguistic tokens, and (iii) a selected token of the ordered sequence of linguistic tokens, responsive to a successful matching.
A parser for parsing a linguistic input, the parser comprising:
- a lexicon;
- a tokenizing module in operative communication with the lexicon, the tokenizing module dividing the linguistic input into an ordered sequence of linguistic tokens;
- and
- a character pattern recognition component for attempting matching of an ordered sequence of characters with a character pattern;
- a token pattern recognition component for attempting matching of the ordered sequence of linguistic tokens with a syntactical pattern, the token pattern recognition component invoking the character pattern recognition component to attempt matching of an ambiguous portion of the ordered sequence of linguistic tokens with an indeterminate portion of the syntactical pattern;
- and
- a category associator for associating a constituent category with at least one of:
- (i) the ordered sequence of linguistic tokens, (ii) an ordered sub-sequence of the ordered sequence of linguistic tokens, and (iii) a selected token of the ordered sequence of linguistic tokens, the associating being performed responsive to a successful matching performed by the token pattern recognition component.
Dependencies are inferred from the claim text.