Construction of a lexicon for a selected context
AssigneeMicrosoft Technology Licensing, LLC
InventorsDarren Chuang, Jingmei Li, Zhen Liu, Mak Chiu Chun Bobby
Life of this patent
Estimating the term…
Abstract
Various technologies pertaining to constructing a lexicon for a defined context are set forth herein. Social media text is acquired, where the social media text has contextual data that corresponds thereto. The social media text is encoded to form encoded text (in Unicode), and the contextual data is assigned to the encoded text. A text corpus for a defined context is formed by filtering the encoded text based upon contextual data, such as location. Frequency of occurrence of words or phrases in the text corpus is used to identify words or phrases that are to be included in the lexicon.
Claims20
A computing system comprising:
- a processor;
- and
- a memory storing instructions that, when executed by the processor, provide a text corpus generator system to:
- identify text generated by network users and associated with a context;
- remove information from the generated text that may identify one or more network users to generate context text;
- and
- a lexicon generator system to:
- identify one or more terms of the context text;
- identify a frequency of each of the one or more terms;
- and
- generate a lexicon associated with the context based on the one or more terms of the context text and the frequency of each of the one or more terms.
A method comprising:
- identifying text generated by users of a social networking application and associated with a context;
- generating context text by removing information from the identified text that may identify one or more users;
- identifying entries in the context text, the entries representative of phrases or sentences in the context text;
- executing a scan over the entries in the context text to form compound words;
- computing a frequency of occurrence of each formed compound word;
- and
- generating a lexicon based upon the frequency of occurrence of the compound words.
A computer-readable storage medium comprising instructions that, when executed by a processor, cause the processor to perform acts comprising:
- accessing text that is identified as corresponding to a particular geographic region, the text based upon posts generated in a social networking application;
- removing information from the accessed text that may identify one or more users of the social networking application to generate context text;
- and
- constructing a lexicon for the particular geographic region based upon frequency of occurrence of words and phrases represented in the context text, wherein constructing the lexicon comprises identifying at least one mixed language word or mixed language phrase represented in the context text to include in the lexicon based upon a frequency of occurrence of the at least one mixed language word or mixed language phrase in the context text.
Dependencies are inferred from the claim text.