# US 8,571,866 B2 · System and method for improving speech recognition accuracy using textual context > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 8,571,866 B2 - Document ID: US8571866B2 - Status: Granted - Kind: B2 - Application number: 12/604,628 - Filed: 2009-10-23 - Published: 2011-04-28 - Granted: 2013-10-29 - Earliest claimed: 2009-10-23 - Assignees: AT&T Intellectual Property I, L.P. - Applicants: Dan Melamed; Srinivas Bangalore; Michael Johnston - Inventors: Dan Melamed; Srinivas Bangalore; Michael Johnston - Web page: https://patentgrep.com/patent/US8571866B2 ## CPC - G10L 25/51: for comparison or discrimination - G06F 3/162: {Interface to dedicated audio devices, e.g. audio drivers, interface to CODECs} - G10L 15/05: Word boundary detection - G10L 15/07: to the speaker - G10L 15/18: using natural language modelling - G10L 15/19: Grammatical context, e.g. disambiguation of the recognition hypotheses based on word sequence rules - G10L 15/30: Distributed recognition, e.g. in client-server systems, for mobile phones or network applications - G10L 17/04: Training, enrolment or model building - G10L 15/183: using context dependencies, e.g. language models - G10L 2015/228: {of application context} - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Disclosed herein are systems, methods, and computer-readable storage media for improving speech recognition accuracy using textual context. The method includes retrieving a recorded utterance, capturing text from a device display associated with the spoken dialog and viewed by one party to the recorded utterance, and identifying words in the captured text that are relevant to the recorded utterance. The method further includes adding the identified words to a dynamic language model, and recognizing the recorded utterance using the dynamic language model. The recorded utterance can be a spoken dialog. A time stamp can be assigned to each identified word. The method can include adding identified words to and/or removing identified words from the dynamic language model based on their respective time stamps. A screen scraper can capture text from the device display associated with the recorded utterance. The device display can contain customer service data. ## Claims ### Claim 1 A method comprising: retrieving a recorded utterance; retrieving captured text, the captured text having been captured from an image displayed on a device display via a screen scraper and recognized using optical character recognition, wherein the device display is associated with the recorded utterance and the device display presents the captured text to a party associated with the recorded utterance; identifying, via a processor, words in the captured text that are relevant to the recorded utterance, to yield identified words; modifying a dynamic language model by: (1) adding the identified words to the dynamic language model; and (2) removing an existing word from the dynamic language model based on a decay time, wherein the decay time of the existing word is determined based on a prominence of the existing word; and recognizing the recorded utterance using the dynamic language model. ### Claim 2 (depends on 1) The method of claim 1, wherein the recorded utterance is a spoken dialog. ### Claim 3 (depends on 2) The method of claim 2, the method further comprising: identifying a user in the spoken dialog; and saving the dynamic language model as a personalized dynamic language model associated with the user. ### Claim 4 (depends on 3) The method of claim 3, the method further comprising: retrieving a second spoken dialog associated with the user; loading the personalized dynamic language model associated with the user; and recognizing the second spoken dialog using the personalized dynamic language model. ### Claim 5 (depends on 1) The method of claim 1, wherein each identified word in the identified words is assigned a time stamp. ### Claim 6 (depends on 5) The method of claim 5, the method further comprising adding one of the identified words to the dynamic language model based on its respective assigned time stamp. ### Claim 7 (depends on 5) The method of claim 5, the method further comprising removing one of the identified words from the dynamic language model based on its respective assigned time stamp. ### Claim 8 (depends on 1) The method of claim 1, wherein the device display presents customer service data. ### Claim 9 (depends on 1) The method of claim 1, wherein the captured text comprises one of a name, a location, a phone number, an account type, and a product name. ### Claim 10 (depends on 1) The method of claim 1, the method further comprising: determining an utterance category based on the captured text; and adding utterance category specific words to the dynamic language model. ### Claim 11 (depends on 1) The method of claim 1, wherein adding the identified words to the dynamic language model further comprises rescoring an existing language model. ### Claim 12 (depends on 1) The method of claim 1, wherein identifying words in the captured text that are relevant to the recorded utterance further comprises: extracting from the captured text a reference to external data; retrieving the external data; identifying data of interest in the external data; and adding the identified data of interest to the dynamic language model. ### Claim 13 A system comprising: a processor; and a computer readable storage medium having instructions stored which, when executed on the processor, perform operations comprising: retrieving a recorded utterance; retrieving captured text, the captured text having been captured from an image displayed on a device display via a screen scraper and recognized using optical character recognition, wherein the device display is associated with the recorded utterance and the device display presents the captured text to a party associated with the recorded utterance; identifying words in the captured text that are relevant to the recorded utterance, to yield identified words; modifying a dynamic language model by: (1) adding the identified words to the dynamic language model; and (2) removing an existing word from the dynamic language model based on a decay time, wherein the decay time of the existing word is determined based on a prominence of the existing word; and recognizing the recorded utterance using the dynamic language model. ### Claim 14 (depends on 13) The system of claim 13, wherein the recorded utterance is part of a spoken dialog. ### Claim 15 (depends on 13) The system of claim 13, wherein each identified word is assigned a time stamp. ### Claim 16 (depends on 15) The system of claim 15, the computer readable storage medium having additional instructions stored which result in the operations further comprising adding one of the identified words to the dynamic language model based on its respective assigned time stamp. ### Claim 17 A computer-readable storage device having instructions stored which, when executed by a computing device, cause the computing device to perform operations comprising: retrieving a recorded utterance; retrieving captured text, the captured text having been captured from an image displayed on a device display via a screen scraper and recognized using optical character recognition, wherein the device display is associated with the recorded utterance and the device display presents the captured text to a party associated with the recorded utterance; identifying words in the captured text that are relevant to the recorded utterance, to yield identified words; modifying a dynamic language model by: (1) adding the identified words to the dynamic language model; and (2) removing an existing word from the dynamic language model based on a decay time, wherein the decay time of the existing word is determined based on a prominence of the existing word; and recognizing the recorded utterance using the dynamic language model. ### Claim 18 (depends on 17) The computer-readable storage device of claim 17, wherein the recorded utterance is part of a spoken dialog. ### Claim 19 (depends on 17) The computer-readable storage device of claim 17, the computer-readable storage device having additional instructions stored which result in the operations further comprising: extracting from the captured text a reference to external data; retrieving the external data; identifying data of interest in the external data; and adding the identified data of interest to the dynamic language model.