# US 8,543,373 B2 · System for compiling word usage frequencies > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 8,543,373 B2 - Document ID: US8543373B2 - Status: Granted - Kind: B2 - Application number: 12/270,082 - Filed: 2008-11-13 - Published: 2009-03-05 - Granted: 2013-09-24 - Earliest claimed: 2005-01-13 - Assignees: International Business Machines Corporation - Applicants: Yen-Fu Chen; John W. Dunsmoir - Inventors: Yen-Fu Chen; John W. Dunsmoir - Web page: https://patentgrep.com/patent/US8543373B2 ## CPC - G06F 40/216: using statistical methods - G09B 19/06: Foreign languages (with audible presentation of material to be studied G09B5/04) - G10L 15/183: using context dependencies, e.g. language models - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G09B: EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS (devices for psychotechnics or for testing reaction times A61B5/16; games, sports, amusements A63; projectors, projector screens G03B) - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING ## Abstract A system for assisting a user who is learning a language to prioritize words to be learned in order of usage frequency is disclosed. A frequency determination program running on a computer determines the frequency of usage of each word at a list of locations provided by the user. Different algorithms to identify what constitutes a word are employed depending upon the language of the source data. The total number of words at each location and their usage frequency found during the user session, along with a total number of words and their usage frequency for all user sessions performed regardless of location, are calculated and made available to the user. The user can view usage frequencies for words from a single location, a group of locations, or all user sessions performed. ## Claims ### Claim 1 A computer system for managing word usage frequencies, the computer system comprising: one or more processors, one or more computer-readable memories and one or more computer-readable tangible storage devices; program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, to receive: an identifier of a location; an identifier of a number of document levels at the location, the document levels comprising at least one document; and an identifier of a minimum number of words; program instructions, stored on the at least one of the one or more storage devices for execution by the at least one of the one or more processors via the at least one of the one or more memories, to determine whether the minimum number of words are present in the at least one document; program instructions, stored on the at least one of the one or more storage devices for execution by the at least one of the one or more processors via the at least one of the one or more memories, responsive to determining that the minimum number of words are present in the at least one document, to analyze all of the at least one document to determine whether the at least one document comprises Latin based words or Sino-Tibetan based words; program instructions, stored on the at least one of the one or more storage devices for execution by the at least one of the one or more processors via the at least one of the one or more memories, responsive to determining that the at least one document comprises the Latin based words, to populate a Latin based word list with unique Latin based words in the at least one document and to determine a frequency of each of the unique Latin based words in the at least one document; program instructions, stored on the at least one of the one or more storage devices for execution by the at least one of the one or more processors via the at least one of the one or more memories, responsive to determining that the at least one document comprises the Sino-Tibetan based words, to populate a Sino-Tibetan based word list with unique Sino-Tibetan based words in the at least one document and to determine a frequency of the unique Sino-Tibetan based words in the at least one document; and program instructions, stored on the at least one of the one or more storage devices for execution by the at least one of the one or more processors via the at least one of the one or more memories, to generate results comprising the unique Latin based words, the frequency of the unique Latin based words, the unique Sino-Tibetan based words, and the frequency of the unique Sino-Tibetan based words. ### Claim 2 (depends on 1) The computer system of claim 1, wherein the program instructions to determine whether the location comprises the Latin based words or the Sino-Tibetan based words identify a Unicode encoding for words in the location. ### Claim 3 (depends on 1) The computer system of claim 1, wherein the location is a website having hyperlinks. ### Claim 4 (depends on 1) The computer system of claim 1, further comprising program instructions, stored on the at least one of the one or more storage devices for execution by the at least one of the one or more processors via the at least one of the one or more memories, responsive to determining that the at least one document comprises the Latin based words, to calculate a ratio of the frequency of the Latin based words to a total number of the Latin based words. ### Claim 5 (depends on 1) The computer system of claim 1, further comprising program instructions, stored on the at least one of the one or more storage devices for execution by the at least one of the one or more processors via the at least one of the one or more memories, responsive to determining that the at least one document comprises the Sino-Tibetan based words, to calculate a ratio of the frequency of the Sino-Tibetan based words to a total number of the Sino-Tibetan words. ### Claim 6 (depends on 1) The computer system of claim 1, wherein the program instructions to determine the frequency of the unique Sino-Tibetan based words: search a Sino-Tibetan word dictionary for an entry that comprises a sequence of characters in at least one of the unique Sino-Tibetan based words; and in response to determining that the dictionary includes the entry, add the at least one of the unique Sino-Tibetan based words to the Sino-Tibetan based word list. ### Claim 7 (depends on 6) The computer system of claim 6 then comprising: program instructions, stored on the at least one of the one or more storage devices for execution by the at least one of the one or more processors via the at least one of the one or more memories, to convert an encoding of the at least one document from a first encoding to a second encoding, wherein the dictionary is stored in the second encoding. ### Claim 8 (depends on 7) The computer system of claim 7, wherein the dictionary is also stored in another encoding. ### Claim 9 (depends on 1) The computer system of claim 1, further comprising: program instructions, stored on the at least one of the one or more storage devices for execution by the at least one of the one or more processors via the at least one of the one or more memories, to identify a link in the at least one document to at least one other document; program instructions, stored on the at least one of the one or more storage devices for execution by the at least one of the one or more processors via the at least one of the one or more memories, to determine whether the at least one other document is present within the number of document levels at the location; and program instructions, stored on the at least one of the one or more storage devices for execution by the at least one of the one or more processors via the at least one of the one or more memories, to cease processing of the at least one other document if the at least one other document is not present within the number of document levels at the location. ### Claim 10 (depends on 1) The computer system of claim 1, wherein the program instructions to determine the frequency of the each of the unique Latin based words in the at least one document: determine that a unique Latin based word in the at least one document contains at least one leading punctuation mark, and in response, remove the leading punctuation mark from the Latin based word prior to populating the Latin based word list with the unique Latin based words. ### Claim 11 A computer program product for managing word usage frequencies, the computer program product comprising: one or more computer-readable tangible storage devices; program instructions, stored on at least one of the one or more storage devices, to receive: an identifier of a location comprising at least one document, an identifier of a number of document levels at the location, the document levels comprising the at least one document, and an identifier of a minimum number of words; program instructions, stored on at least one of the one or more storage devices, to determine whether the minimum number of words are present in the at least one document; program instructions, stored on at least one of the one or more storage devices, responsive to determining that the minimum number of words are present in the at least one document, to analyze all of the at least one document to determine whether the at least one document comprises Latin based words or Sino-Tibetan based words; program instructions, stored on at least one of the one or more storage devices, responsive to determining that the at least one document comprises the Latin based words, to populate a Latin based word list with unique Latin based words in the at least one document and to determine a frequency of each of the unique Latin based words in the at least one document; program instructions, stored on at least one of the one or more storage devices, responsive to determining that the at least one document comprises the Sino-Tibetan based words, to populate a Sino-Tibetan based word list with unique Sino-Tibetan based words in the at least one document and to determine a frequency of the unique Sino-Tibetan based words in the at least one document; and program instructions, stored on at least one of the one or more storage devices, to generate results comprising the unique Latin based words, the frequency of the unique Latin based words, the unique Sino-Tibetan based words, and the frequency of the unique Sino-Tibetan based words. ### Claim 12 (depends on 11) The computer program product of claim 11, wherein the program instructions to determine whether the location comprises Latin based words or Sino-Tibetan based words identify a Unicode encoding for words in the location. ### Claim 13 (depends on 11) The computer program product of claim 11, wherein the location is a website having hyperlinks, and further comprising program instructions, stored on at least one of the one or more storage devices, responsive to determining that the at least one document comprises the Latin based words, to calculate a ratio of the frequency of the Latin based words to a total number of the Latin based words. ### Claim 14 (depends on 11) The computer program product of claim 11, wherein the program instructions to determine the frequency of the Sino-Tibetan based words: search a Sino-Tibetan word dictionary for an entry that comprises a sequence of characters in at least one of the unique Sino-Tibetan based words; and in response to determining that the dictionary includes the entry, add the at least one of the unique Sino-Tibetan based words to the Sino-Tibetan based word list. ### Claim 15 (depends on 14) The computer program product of claim 14 further comprising: program instructions, stored on at least one of the one or more storage devices, to convert an encoding of the at least one document from a first encoding to a second encoding, wherein the dictionary is stored in the second encoding. ### Claim 16 (depends on 15) The computer program product of claim 15, wherein the dictionary is also stored in another encoding. ### Claim 17 (depends on 11) The computer program product of claim 11, further comprising: program instructions, stored on the at least one of the one or more storage devices to identify a link in the at least one document to at least one other document; program instructions, stored on the at least one of the one or more storage devices to determine whether the at least one other document is present within the number of document levels at the location; and program instructions, stored on the at least one of the one or more storage devices, to cease processing of the at least one other document if the at least one other document is not present within the number of document levels at the location. ### Claim 18 (depends on 11) The computer program product of claim 11, wherein the program instructions to determine the frequency of the each of the unique Latin based words in the at least one document: determine that a unique Latin based word in the at least one document contains at least one leading punctuation mark, and in response, remove the leading punctuation mark from the Latin based word prior to populating the Latin based word list with the unique Latin based words. ### Claim 19 A method for managing word usage frequencies, the method comprising: a computer receiving: an identifier of a location; an identifier of a number of document levels at the location, the document levels comprising at least one document; and an identifier of a minimum number of words; the computer determining whether the minimum number of words are present in the at least one document; responsive to the computer determining that the minimum number of words are present in the at least one document, the computer analyzing all of the at least one document to determine whether the at least one document comprises Latin based words or Sino-Tibetan based words; responsive to the computer determining that the at least one document comprises the Latin based words, the computer populating a Latin based word list with the Latin based words and the computer determining a frequency of the Latin based words; and responsive to the computer determining that the at least one document comprises the Sino-Tibetan based words, the computer populating a Sino-Tibetan based word list with the Sino-Tibetan based words and to determine a frequency of the Sino-Tibetan based words; and the computer generating results comprising the Latin based words, the frequency of the Latin based words, the Sino-Tibetan based words, and the frequency of the Sino-Tibetan based words. ### Claim 20 (depends on 19) The method of claim 19, wherein the computer determining whether the at least one document comprises Latin based words or Sino-Tibetan based words comprises the computer identifying a Unicode encoding for words in the at least one document. ### Claim 21 (depends on 19) The method of claim 19, wherein the location is a website having hyperlinks, and further comprising: responsive to the computer determining that the at least one document comprises the Sino-Tibetan based words, the computer calculating a ratio of the frequency of the Sino-Tibetan based words to a total number of the Sino-Tibetan words. ### Claim 22 (depends on 19) The method of claim 19, wherein the computer determining the frequency of the Sino-Tibetan based words comprises: the computer searching a Sino-Tibetan word dictionary for an entry that comprises a sequence of characters in at least one of the Sino-Tibetan based words; and in response to the computer determining that the dictionary includes the entry, the computer adding the at least one of the Sino-Tibetan based words to the Sino-Tibetan based word list. ### Claim 23 (depends on 22) The method of claim 22 further comprising: the computer converting an encoding of the at least one document from a first encoding to a second encoding, wherein the dictionary is stored in the second encoding. ### Claim 24 (depends on 23) The method of claim 23, wherein the dictionary is also stored in another encoding. ### Claim 25 (depends on 19) The method of claim 19, further comprising: the computer identifying a link in the at least one document to at least one other document; and the computer determining that the at least one other document is not present within the number of document levels at the location, and in response, the computer ceasing processing of the at least one other document. ### Claim 26 (depends on 19) The method of claim 19, wherein the computer determining the frequency of the each of the unique Latin based words in the at least one document comprises: the computer determining that a unique Latin based word in the at least one document contains at least one leading punctuation mark, and in response, the computer removing the leading punctuation mark from the Latin based word prior to populating the Latin based word list with the unique Latin based words.