# US 2004/0083099 A1 · Methods and apparatus for audio data analysis and data mining using speech recognition

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2004/0083099 A1
- Document ID: US20040083099A1
- Status: Published application
- Kind: A1
- Application number: 10/687,703
- Filed: 2003-10-20
- Published: 2004-04-29
- Earliest claimed: 2002-10-18
- Assignees: SER Solutions, Inc.
- Inventors: Robert Scarano; Lawrence Mark
- Web page: https://patentgrep.com/patent/US20040083099A1

## CPC

- G10L 15/08: Speech classification or search
- G06F 16/685: {using automatically derived transcript of audio data, e.g. lyrics (speech recognition G10L15/00)}
- G10L 15/02: Feature extraction for speech recognition; Selection of recognition unit
- H04M 2201/40: using speech recognition
- H04M 2203/301: Management of recordings
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04M: TELEPHONIC COMMUNICATION (circuits for controlling other apparatus via a telephone cable and not involving telephone switching apparatus G08)

## Abstract

The present invention provides an audio analysis intelligence tool that provides ad-hoc search capabilities using spoken words as an organized data form. The present invention provides an SQL like interface to process and search audio data and combine it with other traditional data forms.

## Claims

### Claim 1

A method of searching audio data, comprising the step of:
defining a phrase to use for searching;
defining a minimum confidence level for searching;
searching a set of audio segment for said phrase; and
producing a set of results of all occurrences of the phrase within the audio segments and the confidence that a given occurrence is a match for the search phrase.

### Claim 2 (depends on 1)

The method according to claim 1 wherein said step of defining includes defining a plurality of phrases, said step of searching includes searching said set of audio segments for said plurality of phrases, and said step of producing includes producing a set of results of all occurrences of the plurality of phrases identified in a specified sequential order within the audio segments with said minimum confidence that a given occurrence within said audio segments is a match for a corresponding one of said plurality of search phrases.

### Claim 3 (depends on 1)

The method according to claim 1 wherein said step of defining includes defining a plurality of phrases, said step of searching includes searching said set of audio segments for said plurality of phrases, and said step of producing includes producing a set of results of all audio segments including (i) at least one occurrence of a selected required one of the plurality of phrases and (ii) non-occurrences of at least one selected forbidden one of said plurality of phrases to be excluded from within the audio segments, said occurrence and non-occurrence determined with respect to said minimum confidence that a given occurrence within said audio segments is a match for a corresponding one of said plurality of search phrases.

### Claim 4 (depends on 1)

The method according to claim 1 wherein said step of defining includes defining a plurality of phrases, said step of searching includes searching said set of audio segments for said plurality of phrases, and said step of producing includes producing a set of results of all occurrences of the plurality of phrases identified in a specified temporal relationship within the audio segments with said minimum confidence that a given occurrence within said audio segments is a match for a corresponding one of said plurality of search phrases.

### Claim 5 (depends on 1)

The method according to claim 1 wherein said step of defining includes defining a plurality of phrases, said step of searching includes searching said set of audio segments for said plurality of phrases, and said step of producing includes producing a set of results of all audio segments lacking occurrences of the plurality of phrases identified in a specified temporal relationship within the audio segments with said minimum confidence that a given occurrence within said audio segments is a match for a corresponding one of said plurality of search phrases.

### Claim 6 (depends on 5)

The method according to claim 5 wherein said temporal relationship is with respect to said phrases.

### Claim 7 (depends on 5)

The method according to claim 5 wherein said temporal relationship is with respect to said audio segment.

### Claim 8 (depends on 1)

The method according to claim 1 wherein said step of defining includes defining a plurality of phrases, said step of searching includes searching said set of audio segments for said plurality of phrases, and said step of producing includes producing a set of results of all occurrences of the plurality of phrases identified in a specified temporal relationship within the audio segments with said minimum confidence that a given occurrence within said audio segments is a match for a corresponding one of said plurality of search phrases.

### Claim 9 (depends on 8)

The method according to claim 8 wherein said temporal relationship is with respect to said phrases.

### Claim 10 (depends on 8)

The method according to claim 8 wherein said temporal relationship is with respect to said audio segment.

### Claim 11 (depends on 1)

The method according to claim 1 further comprising the step of identifying said set of audio segments.

### Claim 12 (depends on 11)

The method according to claim 11 wherein said step of identifying is responsive to intrinsic data.

### Claim 13 (depends on 11)

The method according to claim 11 wherein said step of identifying is response to CTI data.

### Claim 14 (depends on 13)

The method according to claim 13 wherein said CTI data selected from the set consisting of (i) called number (DNIS) and, calling number (ANI), and (iii) Agent ID.

### Claim 15

A method of operating contact center, comprising the step of:
connecting a plurality of calls to at least one customer service representative;
recording audio segments from each of said plurality of calls;
defining a phrase to use for searching;
defining a minimum confidence level for searching;
searching said set of audio segment for said phrase; and
producing a set of results of all occurrences of the phrase within the audio segments and the confidence that a given occurrence is a match for the search phrase.

### Claim 16

A system for searching audio data comprising:
control logic operable to define a phrase to use for searching and define a minimum confidence level for searching; and
a search engine operable to search a set of audio segment for said phrase and produce a set of results of all occurrences of the phrase within the audio segments and the confidence that a given occurrence is a match for the search phrase.

### Claim 17 (depends on 16)

The system according to claim 16 wherein said control logic is further operable to define a plurality of phrases, said search engine further operable to search said set of audio segments for said plurality of phrases and produce a set of results of all occurrences of the plurality of phrases identified in a specified sequential order within the audio segments with said minimum confidence that a given occurrence within said audio segments is a match for a corresponding one of said plurality of search phrases.

### Claim 18 (depends on 16)

The system according to claim 16 wherein said control logic is further operable to define a plurality of phrases, said search engine further operable to search said set of audio segments for said plurality of phrases and said produce a set of results of all audio segments including (i) at least one occurrence of a selected required one of the plurality of phrases and (ii) non-occurrences of at least one selected forbidden one of said plurality of phrases to be excluded from within the audio segments, said occurrence and nonoccurrence determined with respect to said minimum confidence that a given occurrence within said audio segments is a match for a corresponding one of said plurality of search phrases.

### Claim 19 (depends on 16)

The system according to claim 16 wherein said control logic is further operable to define a plurality of phrases, said search engine further operable to search said set of audio segments for said plurality of phrases and produce a set of results of all occurrences of the plurality of phrases identified in a specified temporal relationship within the audio segments with said minimum confidence that a given occurrence within said audio segments is a match for a corresponding one of said plurality of search phrases.

### Claim 20 (depends on 16)

The system according to claim 16 wherein said control logic is operable to define a plurality of phrases, said search engine further operable to search said set of audio segments for said plurality of phrases and produce a set of results of all audio segments lacking occurrences of the plurality of phrases identified in a specified temporal relationship within the audio segments with said minimum confidence that a given occurrence within said audio segments is a match for a corresponding one of said plurality of search phrases.

### Claim 21 (depends on 20)

The system according to claim 20 wherein said temporal relationship is with respect to said phrases.

### Claim 22 (depends on 20)

The system according to claim 20 wherein said temporal relationship is with respect to said audio segment.

### Claim 23 (depends on 16)

The system according to claim 16 wherein said control logic is operable to define a plurality of phrases, said search engine operable to search said set of audio segments for said plurality of phrases and produce a set of results of all occurrences of the plurality of phrases identified in a specified temporal relationship within the audio segments with said minimum confidence that a given occurrence within said audio segments is a match for a corresponding one of said plurality of search phrases.

### Claim 24 (depends on 23)

The system according to claim 23 wherein said temporal relationship is with respect to said phrases.

### Claim 25 (depends on 23)

The system according to claim 23 wherein said temporal relationship is with respect to said audio segment.

### Claim 26 (depends on 16)

The system according to claim 16 wherein said processor is further operable to identify said set of audio segments.

### Claim 27 (depends on 26)

The system according to claim 26 wherein said processor is responsive to CTI data for identifying said set of audio segments.

### Claim 28 (depends on 27)

The system according to claim 27 wherein said CTI data selected from the set consisting of (i) called number (DNIS), (ii) calling number (ANI), and (iii) Agent ID.

### Claim 29

A contact center comprising:
a switch configured to connect each of a plurality of calls to a customer service representative workstation;
a memory connected to said switch and configured to record audio segments from each of said plurality of calls;
a supervisory terminal configured to define a phrase to use for searching and a minimum confidence level for searching;
a search engine connected to said supervisory terminal and to said memory for searching said set of audio segment for said phrase; and
a display connected to said search engine and configured to produce a set of results of all occurrences of the phrase within the audio segments and the confidence that a given occurrence is a match for the search phrase.

### Claim 30

A method for analyzing audio data, comprising:
storing an audio segment in a speech repository;
storing information regarding the audio segment in a database;
establishing a search criteria including speech and SQL criteria for locating for spoken words or phrases in said audio segment using speech recognition technology;
searching said audio segment and said database in accordance with said search criteria; and
providing a report based on said search.
