Construction of a lexicon for a selected context
AssigneeMicrosoft Technology Licensing, LLC
InventorsDarren Chuang, Jingmei Li, Zhen Liu, Mak Chiu Chun Bobby
Life of this patent
Estimating the term…
Abstract
Various technologies pertaining to constructing a lexicon for a defined context are set forth herein. Social media text is acquired, where the social media text has contextual data that corresponds thereto. The social media text is encoded to form encoded text (in Unicode), and the contextual data is assigned to the encoded text. A text corpus for a defined context is formed by filtering the encoded text based upon contextual data, such as location. Frequency of occurrence of words or phrases in the text corpus is used to identify words or phrases that are to be included in the lexicon.
Claims20
A computing system comprising:
- a processor;
- and
- memory that comprises a lexicon generator system that is executed by the processor, the lexicon generator system configured to generate a lexicon used in context text, the lexicon generator system configured to include at least one of a mixed language word or a mixed language phrase in the lexicon based upon frequency of occurrence of the mixed language word or the mixed language phrase in the context text.
A method executed by a computing system that is configured to generate a lexicon for a predefined context, the method comprising:
- accessing context text that corresponds to the predefined context, the context text acquired from at least one computer-implemented social networking application, the context text comprises at least one mixed language word or mixed language phrase;
- identifying entries in the context text, the entries representative of phrases or sentences in the context text;
- executing a scan over the entries in the context text to form compound words, a compound word representative of one of the mixed language word or the mixed language phrase;
- computing a frequency of occurrence of the compound word;
- and
- generating the lexicon based upon the frequency of occurrence of the compound word, the lexicon comprises the mixed language word or the mixed language phrase represented by the compound word.
A computer-readable storage medium comprising instructions that, when executed by a processor, cause the processor to perform acts comprising:
- accessing context text that is identified as corresponding to a particular geographic region, the context text based upon posts set forth in a social networking application;
- and
- constructing a lexicon for the particular geographic region based upon frequency of occurrence of words and phrases represented in the context text, wherein constructing the lexicon comprises identifying at least one mixed language word or mixed language phrase represented in the context text to include in the lexicon based upon a frequency of occurrence of the at least one mixed language word or mixed language phrase in the context text.
Dependencies are inferred from the claim text.