# US 2014/0156260 A1 · Generating sentence completion questions

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2014/0156260 A1
- Document ID: US20140156260A1
- Status: Published application
- Kind: A1
- Application number: 13/690,424
- Filed: 2012-11-30
- Published: 2014-06-05
- Earliest claimed: 2012-11-30
- Assignees: Microsoft Corporation
- Applicants: Microsoft Corporation
- Inventors: Geoffrey G. Zweig; Christopher J.C. Burges
- Web page: https://patentgrep.com/patent/US20140156260A1

## CPC

- G06F 40/237: Lexical tools
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

The subject disclosure is directed towards automated processes for generating sentence completion questions based at least in part on a language model. Using the language model, a sentence is located, and alternates for a focus word (or words) in the sentence are automatically provided. Also described is automated filtering candidate sentences to locate the sentence, filtering the alternates based upon elimination criteria, scoring sentences with the correct word and as modified the alternates, and ranking the alternates. Manual selection may be used along with the automated processes.

## Claims

### Claim 1

A method performed at least in part on at least one processor, comprising, generating a sentence completion question based at least in part on a language model, including locating a sentence based upon data processed in association with the language model, and providing alternates based upon data processed from the language model.

### Claim 2 (depends on 1)

The method of claim 1 wherein locating the sentence comprises receiving an input focus word and accessing a data store of sentences to retrieve sentences containing the focus word.

### Claim 3 (depends on 2)

The method of claim 2 further comprising, computing a probability score for the sentence containing the focus word, and computing a probability score for each modified sentence in which the focus word is replaced with an alternate.

### Claim 4 (depends on 2)

The method of claim 2 further comprising, ranking the sentences based upon data processed from the language model.

### Claim 5 (depends on 1)

The method of claim 1 wherein providing the alternates comprises generating a plurality of candidate alternates from which the alternates are selectable.

### Claim 6 (depends on 5)

The method of claim 5 further comprising, ranking the candidate alternates based upon data processed from the language model.

### Claim 7 (depends on 5)

The method of claim 5 further comprising, filtering the candidate alternates.

### Claim 8 (depends on 7)

The method of claim 7 wherein filtering the candidate alternates comprises performing automated synonym filtering.

### Claim 9 (depends on 7)

The method of claim 7 wherein filtering the candidate alternates comprises performing automated part-of-speech filtering.

### Claim 10 (depends on 7)

The method of claim 7 wherein filtering the candidate alternates comprises performing automated conjugation filtering.

### Claim 11

In a computing environment, a system comprising, one or more processors, a sentence filter implemented on the one or more processors, the sentence filter configured to select a sentence for a sentence completion question, and an alternates generator implemented on the one or more processors, the alternates generator configured to provide a plurality of candidate alternates from which selected alternates are obtained, the selected alternates and a correct answer being output as a set of options for completing the sentence completion question.

### Claim 12 (depends on 11)

The system of claim 11 wherein the sentence filter accesses an N-gram language model to select the sentence.

### Claim 13 (depends on 11)

The system of claim 11 wherein the alternates generator accesses an N-gram language model to provide the candidate alternates.

### Claim 14 (depends on 11)

The system of claim 11 further comprising an alternates filter implemented on the one or more processors, the alternates filter configured to eliminate one or more candidate alternates based upon evaluating the candidate alternates against one or more elimination criteria.

### Claim 15 (depends on 14)

The system of claim 14 wherein the one or more elimination criteria comprises at least one of synonym data, part-of-speech data, or conjugation data.

### Claim 16 (depends on 11)

The system of claim 11 further comprising an alternates ranker implemented on the one or more processors, the alternates ranker configured to rank candidate alternates based upon based upon probability data determined from an N-gram language model.

### Claim 17 (depends on 11)

The system of claim 11 wherein the set of options comprises a single correct answer and two or more alternates.

### Claim 18 (depends on 11)

The system of claim 11 wherein the sentence completion question comprises a sentence with a single word left blank for completion with a word selected from among the set of options.

### Claim 19

One or more computer-readable media having computer-executable instructions, which when executed perform steps, comprising, obtaining candidate sentences for a sentence completion question;
filtering the candidate sentences into a selected sentence based at least in part on a language model;
generating a plurality of candidate alternates for the selected sentence based at least in part on the language model;
filtering the candidate alternates;
ranking the candidate alternates; and
outputting the sentence completion question comprising the selected sentence with a removed word set comprising one or more removed words and a set of options for completing the sentence, the set of options comprising the removed word set and selected alternates chosen from among the candidate alternates.

### Claim 20 (depends on 19)

The one or more computer-readable media of claim 19 wherein ranking the candidate alternates comprises using probability information based at least in part on the language model.
