Region-matching transducers for text-characterization

A computing apparatus for characterizing text using a set of text-characterizations, comprising:

• a memory for recording (a) computer readable instructions for performing text-characterization, (b) input data and (c) a finite state transducer that along each path (i) accepts on a first side an n-gram representing a text-characterization and (ii) outputs on a second side a sequence of symbols identifying one or more text-characterizations from the set of text-characterizations;
• a processor for processing the computer readable instructions;
• wherein the processor in processing the computer readable instructions:
• (a) applies the finite state transducer to the input data to identify n-grams of text-characterization in the input data accepted by the finite state transducer on the first side;
• (b) for each n-gram accepted by the finite state transducer on the first side, increments a frequency counter for each of the one or more text-characterizations in the set of text-characterizations, the frequency counter specific to the text-characterization;
• (c) assigns the input data one or more text-characterizations from the set of text-characterizations using the frequency counters associated therewith.

A computing apparatus for characterizing text using a set of text-characterizations, comprising:

• a memory for recording (a) computer readable instructions for performing text-characterization, (b) input data and (c) a finite state transducer that along each path (i) accepts on a first side an n-gram representing a text-characterization and (ii) outputs on a second side a sequence of symbols identifying one or more text-characterizations from the set of text-characterizations;
• an FST engine (a) for applying the finite state transducer to the input data to identify n-grams of text-characterization in the input data accepted by the finite state transducer on the first side;
• (b) for each n-gram accepted by the finite state transducer on the first side, incrementing a frequency counter for each of the one or more text-characterizations in the set of text-characterizations, the frequency counter specific to the text-characterization;
• and (c) assigning the input data one or more text-characterizations from the set of text-characterizations using the frequency counters associated therewith, wherein the values of the one or more frequency counters of the one or more text-characterizations are greater than the values of the frequency counters of the other text-characterizations of the set of text-characterizations.

An article of manufacture comprising computer usable media including computer readable instructions embedded therein that causes a computer to perform a method for characterizing text using a set of text-characterizations, wherein the method comprises:

• recording (a) input data and (b) a finite state transducer that along each path (i) accepts on a first side an n-gram representing a text-characterization and (ii) outputs on a second side a sequence of symbols identifying one or more text-characterizations from the set of text-characterizations;
• applying the finite state transducer to the input data to identify n-grams of text-characterization in the input data accepted by the finite state transducer on the first side;
• for each n-gram accepted by the finite state transducer on the first side, incrementing a frequency counter for each of the one or more text-characterizations in the set of text-characterizations, the frequency counter specific to the text-characterization;
• assigning the input data one or more text-characterizations from the set of text-characterizations using the frequency counters associated therewith.