# US 2012/0084312 A1 · Choosing recognized text from a background environment

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2012/0084312 A1
- Document ID: US20120084312A1
- Status: Published application
- Kind: A1
- Application number: 12/896,507
- Filed: 2010-10-01
- Published: 2012-04-05
- Earliest claimed: 2010-10-01
- Assignees: Google Inc.
- Inventors: Scott Jenson
- Web page: https://patentgrep.com/patent/US20120084312A1

## CPC

- G06F 16/90324: {using system suggestions}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A computer-implemented method includes receiving an electronic representation of one or more human voices, recognizing words in a first portion of the electronic representation of the one or more human voices, and sending suggested search terms to a display device for display to a user in a text format. The suggested search terms are based on the recognized words in the first portion of the electronic representation of the one or more human voices. A search query is received from the user, which includes one or more of the suggested search terms that were displayed to the user.

## Claims

### Claim 1

A computer-implemented method comprising:
receiving an electronic representation of one or more human voices;
recognizing words in a first portion of the electronic representation of the one or more human voices;
sending suggested search terms to a display device for display to a user in a text format, wherein the suggested search terms are based on the recognized words in the first portion of the electronic representation of the one or more human voices;
receiving, from the user, a search query that includes one or more of the suggested search terms that were displayed to the user.

### Claim 2 (depends on 1)

The method of claim 1, wherein the electronic representation of the one or more human voices includes an audio file.

### Claim 3 (depends on 2)

The method of claim 2, further comprising receiving an indication from the user that the first portion of the electronic representation begins at a first time of the audio file and ends a second time of the audio file.

### Claim 4 (depends on 1)

The method of claim 1, wherein the first portion of the electronic representation of the one or more human voices includes a portion of a continuous stream of audio data.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
selecting suggested search terms to be sent to the display device based, at least in part, on a value of the search terms for searching for information within a corpus of documents.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
selecting suggested search terms to be sent to the display device based, at least in part, on a frequency, in the recognized words, of the word(s) upon which the suggested search terms are based.

### Claim 7 (depends on 1)

The method of claim 1, further comprising:
selecting suggested search terms to be sent to the display device based, at least in part, on an inverse of a frequency of the search terms in a corpus of documents.

### Claim 8 (depends on 1)

The method of claim 1, further comprising:
selecting suggested search terms to be sent to the display device based, at least in part, on whether or not the search terms are proper nouns.

### Claim 9 (depends on 1)

The method of claim 1, wherein sending the suggested search terms to the display device for display to a user in a text format includes sending the suggested search terms for display in a hypertext format, such that the user may select one or more of the displayed suggested search terms to formulate a search query.

### Claim 10 (depends on 9)

The method of claim 9, wherein sending the suggested search terms for display in the hypertext format, includes sending the suggested search terms for display on a touchscreen in a format such that the user may select a displayed suggested search terms to formulate a search query by tapping on the displayed suggested search terms.

### Claim 11 (depends on 1)

The method of claim 1,
wherein the electronic representation of the one or more human voices is generated by a mobile computing device,
wherein receiving the electronic representation of the one or more human voices includes receiving the electronic representation at a server computer, and
wherein recognizing the words in the electronic representation includes recognizing the words through operation of a voice recognition operation performed at the server computer.

### Claim 12 (depends on 1)

The method of claim 1, wherein sending the suggested search terms to a display device for display to a user in a text format includes sending the suggested search terms for display in a format in which a first text format that includes indicia used to convey a significance of a displayed suggested search term for use as a query term.

### Claim 13 (depends on 1)

The method of claim 1, wherein sending the suggested search terms to a display device for display to a user in a text format includes sending the suggested search terms for display in a format in which a first text format that includes indicia used to convey when a word in the suggested search term was uttered by the one or more human voices.

### Claim 14 (depends on 1)

The method of claim 1, further comprising receiving at a server computing device a search query from the user, the search query including at least one of the suggested search terms.

### Claim 15 (depends on 14)

The method of claim 14, further comprising:
executing the search query; and
returning results of the executed query to the user.

### Claim 16 (depends on 1)

The method of claim 1, further comprising:
recognizing words in a second portion of the electronic representation of the one or more human voices, wherein the second portion is different from the first portion; and
sending suggested search terms to a display device for display to a user in a text format, wherein the suggested search terms are based on the recognized words in the second portion of the electronic representation of the one or more human voices.

### Claim 17 (depends on 16)

The method of claim 16, wherein the first portion and the second portion have a duration of a predetermined time period.

### Claim 18 (depends on 17)

The method of claim 17, further comprising determining the duration of the predetermined time period based on input from the user.

### Claim 19 (depends on 16)

The method of claim 16, wherein the first portion and the second portion include a predetermined number of recognized words.

### Claim 20 (depends on 19)

The method of claim 19, further comprising determining the predetermined number of recognized based on input from the user.

### Claim 21 (depends on 16)

The method of claim 16, wherein the first portion has a duration from a first time to a second time, and wherein the second portion has a duration from third time to a fourth time, the third time being after the first time and before the second time.

### Claim 22 (depends on 16)

The method of claim 16, further comprising:
recognizing words in a series of different portions of the electronic representation of the one or more human voices,
sending a series of groups of suggested search terms to a display device for display to a user in a text format, wherein the groups of suggested search terms are based the recognized words in the different portions of the electronic representation of the one or more human voices.

### Claim 23

A computing system comprising:
a microphone configured to generate an electronic representation of one or more human voices;
a transmitter configured to transmit a first portion of the electronic representation of the one or more human voices to a server computing system;
a receiver configured to receive suggested search terms from the server computing device, wherein the suggested search terms are based on the words recognized through a voice recognition operation performed at the server computing system in the first portion of the electronic representation of the one or more human voices;
a display device configured to display the suggested search query terms to a user in a text format; and
a user input interface engine configured for receiving, from the user, a search query that includes one or more of the suggested search terms that were displayed to the user.

### Claim 24 (depends on 23)

The system of claim 23, wherein the electronic representation of the one or more human voices includes an audio file.

### Claim 25 (depends on 24)

The system of claim 24, wherein the user input interface engine is further configured for receiving an indication from the user that the first portion of the electronic representation begins at a first time of the audio file and ends a second time of the audio file.

### Claim 26 (depends on 23)

The system of claim 23, wherein the first portion of the electronic representation of the one or more human voices includes a portion of a continuous stream of audio data.

### Claim 27 (depends on 23)

The system of claim 23, wherein the suggested search terms received from the server computing device are based, at least in part, on a value of the search terms for searching for information within a corpus of documents.

### Claim 28 (depends on 23)

The system of claim 23, wherein the suggested search terms received from the server computing device are based, at least in part, on a frequency, in the recognized words, of the word(s) upon which the suggested search terms are based.

### Claim 29 (depends on 23)

The system of claim 23, wherein the suggested search terms received from the server computing device are based, at least in part, on an inverse of a frequency of the search terms in a corpus of documents.

### Claim 30 (depends on 23)

The system of claim 23, wherein the suggested search terms received from the server computing device are based, at least in part, on whether or not the search terms are proper nouns.

### Claim 31 (depends on 23)

The system of claim 23, wherein the text format includes a hypertext format, such that the user may select one or more of the displayed suggested search terms to formulate a search query.

### Claim 32 (depends on 31)

The system of claim 31, wherein the display includes a touchscreen through which the user may select a suggested search term displayed in a hypertext format to formulate a search query by tapping on the displayed suggested search term.

### Claim 33 (depends on 23)

The system of claim 23, wherein the display is configured to display suggested search terms in a first text format that includes indicia used to convey a significance of a displayed suggested search term for use as a query term.

### Claim 34 (depends on 23)

The system of claim 23, wherein the display is configured to display suggested search terms in a first text format that includes indicia used to convey when a word in the suggested search term was uttered by the one or more human voices.

### Claim 35 (depends on 23)

The system of claim 23, wherein the transmitter is further configured to transmit to the server computing system a search query that includes at least one of the suggested search terms.

### Claim 36

A server computing system comprising:
a receiver configured to receive an electronic representation of one or more human voices;
a voice recognition engine configured to recognize words in a first portion of the electronic representation of the one or more human voices;
a query term generation engine configured to generate suggested search query terms based on the recognized words in the first portion of the electronic representation of the one or more human voices;
a transmitter configured to send the suggested search terms to a client computing device for display to a user in a text format on a display device; and
a query execution engine configured to receive, from the user, a search query that includes one or more of the suggested search terms that were displayed to the user and to query a database for information matching the search query terms received from the user.

### Claim 37 (depends on 36)

The server computing system of claim 36, wherein the electronic representation of the one or more human voices includes an audio file.

### Claim 38 (depends on 36)

The server computing system of claim 36, wherein the first portion of the electronic representation of the one or more human voices includes a portion of a continuous stream of audio data.

### Claim 39 (depends on 36)

The server computing system of claim 36, wherein the suggested search terms that are generated and transmitted to the client computing device are based, at least in part, on a value of the search terms for searching for information within a corpus of documents.

### Claim 40 (depends on 36)

The server computing system of claim 36, wherein the suggested search terms that are generated and transmitted to the client computing device are based, at least in part, on a frequency, in the recognized words, of the word(s) upon which the suggested search terms are based.

### Claim 41 (depends on 36)

The server computing system of claim 36, wherein the suggested search terms that are generated and transmitted to the client computing device are based, at least in part, on an inverse of a frequency of the search terms in a corpus of documents.

### Claim 42 (depends on 36)

The server computing system of claim 36, wherein the suggested search terms that are generated and transmitted to the client computing device are based, at least in part, on whether or not the search terms are proper nouns.

### Claim 43 (depends on 36)

The server computing system of claim 36, further comprising:
a storage medium configured to store an index of a corpus of documents; and
wherein the query execution engine is configured to query the index for information matching the search query terms received from the user.
