# US 2016/0253313 A1 · Updating language databases using crowd-sourced input

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2016/0253313 A1
- Document ID: US20160253313A1
- Status: Published application
- Kind: A1
- Application number: 14/634,654
- Filed: 2015-02-27
- Published: 2016-09-01
- Earliest claimed: 2015-02-27
- Assignees: Nuance Communications, Inc.
- Applicants: Nuance Communications, Inc.
- Inventors: Christopher Breske; Ethan Bradford; David Field; Wendy Bannister
- Web page: https://patentgrep.com/patent/US20160253313A1

## CPC

- G06F 40/242: Dictionaries
- G06F 40/166: Editing, e.g. inserting or deleting
- G06F 40/169: Annotation, e.g. comment data or footnotes
- G06F 40/274: Converting codes to words; Guess-ahead of partial word inputs
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Technology is described for refining a language model for a language recognition system based on aggregating and analyzing word tag metadata from multiple users of the language. The technology allows a user to mark a word or phrase in a selected language (e.g., as offensive or misspelled, or as a part of speech or other category), combines information collected from multiple users of the selected language, and updates the user's language model based on the combined information from multiple users of the selected language.

## Claims

### Claim 1

A computer-readable memory having contents configured to cause at least one computer having a processor to perform a method for refining a language model used by language recognition systems, the method comprising:
providing a language model for a selected language to various computing devices of multiple users of the selected language;
offering an interface for user input to delete a word from the language model;
receiving a user's input to delete a word from the language model;
in response to the input to delete a word from the language model, prompting the user to provide information about the deletion;
receiving information about the deletion, in response to the prompting;
collecting, from the various computing devices, the received information about the multiple users' deletions of words in the selected language;
aggregating the collected information from the various computing devices about the deletions;
generating updates to the language model based on the aggregated information about the deletions; and
providing the generated language model updates to the various computing devices.

### Claim 2 (depends on 1)

The computer-readable memory of claim 1 wherein providing the language model for the selected language includes building a language model with crowd-sourced vocabulary.

### Claim 3 (depends on 1)

The computer-readable memory of claim 1 wherein prompting the user to provide information about the deletion includes prompting the user to classify the deleted word, and wherein prompting the user to classify the deleted word includes prompting the user to indicate whether the deleted word is at least one of a misspelling, offensive, profanity, sensitive, pornography-related, jargon, technical vocabulary, or uncommon.

### Claim 4 (depends on 1)

The computer-readable memory of claim 1 wherein prompting the user to provide information about the deletion includes prompting the user to classify the deleted word.

### Claim 5 (depends on 1)

The computer-readable memory of claim 1 wherein offering a user input interface to delete a word from the language model and prompting the user to provide information about the deletion include enabling the user to use one of multiple deletion options, such that different deletion options provide different information about the deletion.

### Claim 6 (depends on 1)

The computer-readable memory of claim 1 wherein the information about the deletion is a reason for the deletion.

### Claim 7 (depends on 1)

The computer-readable memory of claim 1 wherein aggregating the collected information from the various computing devices about the deletions includes determining whether a threshold number or percentage of users provided consistent information about the deletions, and wherein requiring a threshold number or percentage of users includes setting a threshold based on the number of the multiple users of the selected language who delete a word from the language model or based on the number of the multiple users of the selected language who provide information about their deletion of a word from the language model.

### Claim 8 (depends on 1)

The computer-readable memory of claim 1 wherein aggregating the collected information from the various computing devices about the deletions includes determining whether a threshold number or percentage of users provided consistent information about the deletions, and aggregating collected information associated with a common geographic location.

### Claim 9 (depends on 1)

The computer-readable memory of claim 1 wherein generating updates to the language model includes classifying a word, modifying weighting of a word, determining whether to include a word in suggestions provided by a language recognition system, or assigning a word to a list of objectionable or blacklisted words.

### Claim 10 (depends on 1)

The computer-readable memory of claim 1, further comprising offering an option for a user to include or exclude a classification of words in the language model on a computing device of the user, and wherein offering an option for a user to include or exclude a classification of words includes offering whether to enable a language recognition system to recognize or suggest words classified as profanity.

### Claim 11 (depends on 1)

The computer-readable memory of claim 1, further comprising offering an option for a user to include or exclude a classification of words in the language model on a computing device of the user.

### Claim 12

A method in a computing system of assisting in refining a language model used by a language recognition system to predict words in a language, the method comprising:
receiving a selection of a word in the language by a user of the computing system;
providing an interface on the computing system for the user to associate at least one category with the selected word;
receiving, via the interface, user input associating a category with the selected word;
recording the received category association information in a language model on the computing system;
transmitting the recorded category association information to a remote computer; and
receiving updates to the language model on the computing system based on additional information associating one or more words in the language with one or more categories by other users of the language.

### Claim 13 (depends on 12)

The method of claim 12 wherein the selection is a word suggested by the language recognition system based on the language model.

### Claim 14 (depends on 12)

The method of claim 12 wherein a word includes an n-gram.

### Claim 15 (depends on 12)

The method of claim 12 wherein receiving a selection of a word includes receiving input to delete the word from the language model or add the word to the language model.

### Claim 16 (depends on 12)

The method of claim 12 wherein receiving user input associating a category with the selected word includes receiving user input identifying the word as profanity, out-of-language, or misspelled.

### Claim 17 (depends on 12)

The method of claim 12, further comprising providing a set of categories from which the user can associate a category with the selected word, wherein a category or an order of categories is determined based on category association user input from other users.

### Claim 18 (depends on 12)

The method of claim 12, further comprising determining by the language recognition system whether to predict a word in the language in response to user input, based on whether the word is associated with a category in the updated language model including the additional information from other users of the language.

### Claim 19

A system for assisting in refining a language model used by a language recognition system to predict words in a language, the system comprising:
at least one memory storing computer-executable instructions of:
a component configured to receive user language model events from each of multiple computing devices;
wherein a plurality of language model events include user metadata classifying a word;
a component configured to aggregate, in a crowd-sourced language model, the metadata about the classifications of the words in the language on the multiple computing devices;
a component configured to compare aggregated metadata classification statistics to one or more thresholds;
a component configured to classify a word in the crowd-sourced language model based on the comparison meeting one or more thresholds;
a component configured to generate updates to the crowd-sourced language model based on the classification of a word in the language; and
a component configured to provide the generated updates to each of the multiple computing devices; and
at least one processor for executing the computer-executable instructions stored in the at least one memory.

### Claim 20 (depends on 19)

The system of claim 19, wherein the component configured to classify a word in the crowd-sourced language model is configured to include a word in one or more of a complete word list, a whitelist of basic vocabulary not to be deleted, a blacklist of misspellings and other intrinsic errors, a list of offensive words, a list of profanity, a sensitive word list to allow but avoid suggesting to users, a list of pornography-related words, lists of jargon and technical vocabulary, or a list of uncommon words.
