Region-Matching Transducers for Text-Characterization

A computing apparatus, comprising:

• a memory for recording (a) computer readable instructions for performing text-characterization, (b) input data and (c) a finite state transducer that along each path (i) accepts on a first side an n-gram representing a text-characterization and (ii) outputs on a second side a sequence of symbols identifying one or more text-characterizations from a set of text-characterizations;
• a processor for processing the computer readable instructions;
• wherein the processor in processing the computer readable instructions:
• (a) applying the finite state transducer to the input data to identify n-grams of text-characterization in the input data accepted by the finite state transducer on the first side;
• for each n-gram accepted by the finite state transducer on the first side, incrementing a frequency counter associated with the one or more text-characterizations in the set of text-characterizations;
• (b) assigning the input data one or more text-characterizations from the set of text-characterizations using the frequency counters associated therewith.

A computing apparatus, comprising:

• a memory for recording (a) computer readable instructions for performing text-characterization, (b) input data and (c) a finite state transducer that along each path (i) accepts on a first side an n-gram representing a text-characterization and (ii) outputs on a second side a sequence of symbols identifying one or more text-characterizations from a set of text-characterizations;
• an FST engine (a) for applying the finite state transducer to the input data to identify n-grams of text-characterization in the input data accepted by the finite state transducer on the first side;
• for each n-gram accepted by the finite state transducer on the first side, incrementing a frequency counter associated with the one or more text-characterizations in the set of text-characterizations;
• and (b) assigning the input data one or more text-characterizations from the set of text-characterizations using the frequency counters associated therewith.

An article of manufacture comprising computer usable media including computer readable instructions embedded therein that causes a computer to perform a method, wherein the method comprises:

• recording (a) input data and (b) a finite state transducer that along each path (i) accepts on a first side an n-gram representing a text-characterization and (ii) outputs on a second side a sequence of symbols identifying one or more text-characterizations from a set of text-characterizations;
• applying the finite state transducer to the input data to identify n-grams of text-characterization in the input data accepted by the finite state transducer on the first side;
• for each n-gram accepted by the finite state transducer on the first side, incrementing a frequency counter associated with the one or more text-characterizations in the set of text-characterizations;
• assigning the input data one or more text-characterizations from the set of text-characterizations using the frequency counters associated therewith.