Automated Data Cleanup
AssigneeMicrosoft Technology Licensing, LLC
InventorsGeoffrey G. Zweig, Yun-Cheng Ju
Life of this patent
Estimating the term…
Abstract
The described implementations relate to automated data cleanup. One system includes a language model generated from language model seed text and a dictionary of possible data substitutions. This system also includes a transducer configured to cleanse a corpus utilizing the language model and the dictionary.
Claims20
A system, comprising:
- a language model generated from language model seed text;
- a dictionary of potential data substitutions;
- and,
- a transducer configured to cleanse a corpus utilizing the language model and the dictionary.
A computer-readable storage media having instructions stored thereon that when executed by a computing device cause the computing device to perform acts, comprising:
- applying transduction to a unit of data from a speech recognition training corpus to establish probabilities of potential substitutions on the unit of data;
- and,
- selecting an individual substitution based upon the probabilities sufficient to increase data consistency within the speech recognition training corpus.
A computer-readable storage media having instructions stored thereon that when executed by a computing device cause the computing device to perform acts, comprising:
- acquiring a dictionary that contains text normalization rules;
- obtaining a language model configured from language model seed text;
- and,
- building a transducer configured to clean speech recognition training data utilizing the dictionary and the language model to identify probabilities of potential substitutions for the speech recognition training data.
Dependencies are inferred from the claim text.