# US 2010/0166157 A1 · System and Method for Indexing Voice Mail Messages By Speaker > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2010/0166157 A1 - Document ID: US20100166157A1 - Status: Published application - Kind: A1 - Application number: 12/648,909 - Filed: 2009-12-29 - Published: 2010-07-01 - Earliest claimed: 2000-04-17 - Assignees: AT&T Corp. - Inventors: Julia Hirschberg; Sarangarajan Parthasarathy; Aaron Edward Rosenberg; Stephen Whittaker - Web page: https://patentgrep.com/patent/US20100166157A1 ## CPC - H04M 3/533: Voice mail systems - G10L 17/00: Speaker identification or verification techniques - G10L 17/04: Training, enrolment or model building - H04M: TELEPHONIC COMMUNICATION (circuits for controlling other apparatus via a telephone cable and not involving telephone switching apparatus G08) - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING ## Abstract The invention provides a system and method for indexing and organizing voice mail message by the speaker of the message. One or more speaker models are created from voice mail messages received. As additional messages are left, each of the new messages are compared with existing speaker models to determine the identity of the callers of each of the new messages. The voice mail messages are organized within a user's mailbox by caller. Unknown callers may be identified and tagged by the user and then used to create new speaker models and/or update existing speaker models. ## Claims ### Claim 1 (canceled) ### Claim 2 A method for indexing voicemail messages, the method causing a computing device to perform steps comprising: determining to a threshold certainty an identity of a caller in a stored voicemail message for a voicemail subscriber (1) by comparing speech signals from the voicemail message with a plurality of caller speaker models and (2) based on an analysis of the content of the voicemail message; tagging the voicemail message with the determined identity; and when the identity of the caller of the voicemail message cannot be determined to the threshold certainty: receiving from the voicemail subscriber an indicated identity of the caller; if the indicated identity does not match any known identities, creating a new storage folder for voicemail messages from the caller corresponding to the received indicated identity; and if the received indicated identity matches a known identity, modifying a speaker model associated with the known identity based on speech in the voicemail message. ### Claim 3 (depends on 2) The method of claim 2, wherein each of the plurality of caller speaker models is created from a plurality of voicemail messages left by a same caller. ### Claim 4 (depends on 2) The method of claim 2, wherein the plurality of caller speaker models are created using acoustic features extracted from the voicemail message using speaker recognition techniques. ### Claim 5 (depends on 2) The method of claim 2, wherein determining to the threshold certainty the identity of a caller in a stored voicemail messages further comprises using automatic number identification to assist in determining the identity of the caller. ### Claim 6 (depends on 2) The method of claim 2, wherein determining to the threshold certainty the identity of a caller further comprises using speech recognition techniques to extract caller identity information from the one or more voicemail messages. ### Claim 7 A system for indexing voicemail messages, the system comprising: a processor; a first module controlling the processor to determine to a threshold certainty an identity of a caller in a stored voicemail message for a voicemail subscriber (1) by comparing speech signals from the voicemail message with a plurality of caller speaker models and (2) based on an analysis of the content of the voicemail message; a second module controlling the processor to tag the voicemail message with the determined identity; and when the identity of the caller of the voicemail message cannot be determined to the threshold certainty, a third module controlling the processor to: receive from the voicemail subscriber an indicated identity of the caller; if the indicated identity does not match any known identities, create a new storage folder for voicemail messages from the caller corresponding to the received indicated identity; and if the received indicated identity matches a known identity, modify a speaker model associated with the known identity based on speech in the voicemail message. ### Claim 8 (depends on 7) The system of claim 7, wherein each of the plurality of caller speaker models is created from a plurality of voicemail messages left by a same caller. ### Claim 9 (depends on 7) The system of claim 7, wherein the plurality of caller speaker models are created using acoustic features extracted from the voicemail message using speaker recognition techniques. ### Claim 10 (depends on 7) The system of claim 7, wherein the first module further controls the processor to use automatic number identification to assist in determining the identity of the caller. ### Claim 11 (depends on 7) The system of claim 7, wherein the first module further controls the processor to use speech recognition techniques to extract caller identity information from the one or more voicemail messages. ### Claim 12 A non-transitory computer-readable storage medium storing instructions which, when executed by a computing device, cause the computing device to process voicemail messages, the instructions comprising: determining to a threshold certainty an identity of a caller in a stored voicemail message for a voicemail subscriber (1) by comparing speech signals from the voicemail message with a plurality of caller speaker models and (2) based on an analysis of the content of the voicemail message; tagging the voicemail message with the determined identity; and when the identity of the caller of the voicemail message cannot be determined to the threshold certainty: receiving from the voicemail subscriber an indicated identity of the caller; if the indicated identity does not match any known identities, creating a new storage folder for voicemail messages from the caller corresponding to the received indicated identity; and if the received indicated identity matches a known identity, modifying a speaker model associated with the known identity based on speech in the voicemail message. ### Claim 13 (depends on 12) The non-transitory computer-readable storage medium of claim 12, wherein each of the plurality of caller speaker models is created from a plurality of voicemail messages left by a same caller. ### Claim 14 (depends on 12) The non-transitory computer-readable storage medium of claim 12, wherein the plurality of caller speaker models are created using acoustic features extracted from the voicemail message using speaker recognition techniques. ### Claim 15 (depends on 12) The non-transitory computer-readable storage medium of claim 12, wherein determining to the threshold certainty the identity of a caller in a stored voicemail messages further comprises using automatic number identification to assist in determining the identity of the caller. ### Claim 16 (depends on 12) The non-transitory computer-readable storage medium of claim 12, wherein determining to the threshold certainty the identity of a caller further comprises using speech recognition techniques to extract caller identity information from the one or more voicemail messages.