# US 2013/0259211 A1 · System and method for fingerprinting datasets

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2013/0259211 A1
- Document ID: US20130259211A1
- Status: Published application
- Kind: A1
- Application number: 13/432,838
- Filed: 2012-03-28
- Published: 2013-10-03
- Earliest claimed: 2012-03-28
- Assignees: Interactive Intelligence, Inc.
- Inventors: Kevin Vlack; Felix Immanuel Wyss
- Web page: https://patentgrep.com/patent/US20130259211A1

## CPC

- G06F 16/284: {Relational databases}
- G06F 16/683: using metadata automatically derived from the content
- G10L 19/018: Audio watermarking, i.e. embedding inaudible data in the audio signal
- G10L 25/54: for retrieval
- H03M 7/3059: {Digital compression and data reduction techniques where the original information is represented by a subset or similar information, e.g. lossy compression}
- H03M 7/3066: {by means of a mask or a bit-map}
- H03M 7/3082: {Vector coding (for television signals, see H04N19/94)}
- H04M 3/5158: {in combination with automated outdialling systems (devices for signalling identity of wanted subscriber H04M1/26)}
- H04M 2203/2027: Live party detection
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- H03M: CODING; DECODING; CODE CONVERSION IN GENERAL (using fluidic means F15C4/00; optical analogue/digital converters G02F7/00; coding, decoding or code conversion, specially adapted for particular applications, see the relevant subclasses, e.g. G01D, G01R, G06F, G06T, G09G, G10L, G11B, G11C, H04B, H04L, H04M, H04N; ciphering or deciphering for cryptography or other purposes involving the need for secrecy G09C)
- H04M: TELEPHONIC COMMUNICATION (circuits for controlling other apparatus via a telephone cable and not involving telephone switching apparatus G08)

## Abstract

Systems and methods for the matching of datasets, such as input audio segments, with known datasets in a database are disclosed. In an illustrative embodiment, the use of the presently disclosed systems and methods is described in conjunction with recognizing known network message recordings encountered during an outbound telephone call. The methodologies include creation of a ternary fingerprint bitmap to make the comparison process more efficient. Also disclosed are automated methodologies for creating the database of known datasets from a larger collection of datasets.

## Claims

### Claim 1

A method for identifying a candidate audio segment from an outbound telephone call, the method comprising the steps of:
a) creating a spectrogram of the candidate audio segment;
b) creating a candidate binary acoustic fingerprint bitmap of the spectrogram;
c) comparing the candidate binary acoustic fingerprint bitmap to at least one known binary acoustic fingerprint bitmap of a known network message;
d) if the candidate binary acoustic fingerprint bitmap matches one of said at least one known binary acoustic fingerprint bitmaps within a predetermined threshold, declaring the match; and
e) if the candidate binary acoustic fingerprint bitmap does not match one of said at least one known binary acoustic fingerprint bitmaps within the predetermined threshold, using an answering machine detection algorithm to analyze the candidate audio segment.

### Claim 2 (depends on 1)

The method of claim 1, wherein step (a) comprises transforming the candidate audio segment using a transform selected from the group consisting of: discrete fourier transform, discrete cosine transform, wavelet transform, and filter bank.

### Claim 3 (depends on 1)

The method of claim 1, wherein step (b) comprises the steps of:
b.1) transforming the spectrogram data into Mel space with N+1 frames, where N is an integer;
b.2) computing N-bit binary fingerprint frame values based on band energy differences over successive frames according to the following formula:
F

(
n
,
m
)
=
{
1

if


E

(
n
,
m
)
-
E

(
n
,
m
+
1
)
-
(
E

(
n
-
1
,
m
)
-
E

(
n
-
1
,
m
+
1
)
)
>
0
0

if


E

(
n
,
m
)
-
E

(
n
,
m
+
1
)
-
(
E

(
n
-
1
,
m
)
-
E

(
n
-
1
,
m
+
1
)
)
≤
0
where: E(n,m) is an energy of frequency band m of frame n, and
F(n,m) is an mth bit of fingerprint frame n.

### Claim 4 (depends on 1)

The method of claim 1, wherein step (c) comprises computing a Hamming distance between the candidate binary acoustic fingerprint bitmap and the at least one known binary acoustic fingerprint bitmap.

### Claim 5 (depends on 1)

The method of claim 1, wherein the predetermined threshold is a bit error rate.

### Claim 6

A method for identifying a candidate audio segment from an outbound telephone call, the method comprising the steps of:
a) creating a spectrogram of the candidate audio segment;
b) creating a candidate binary fingerprint bitmap of the spectrogram;
c) comparing the candidate binary fingerprint bitmap to at least one known binary fingerprint bitmap of a known recording;
d) if the candidate binary fingerprint bitmap matches one of said at least one known binary fingerprint bitmaps within a predetermined threshold, declaring the match; and
e) if the candidate binary fingerprint bitmap does not match one of said at least one known binary fingerprint bitmaps within the predetermined threshold, using an alternate process to analyze the candidate audio segment.

### Claim 7 (depends on 6)

The method of claim 6, wherein said candidate binary fingerprint bitmap and said known binary fingerprint bitmap each comprise an acoustic fingerprint bitmap.

### Claim 8 (depends on 6)

The method of claim 6, wherein said recording comprises a network message.

### Claim 9 (depends on 6)

The method of claim 6, wherein said alternate process comprises a process selected from the group consisting of: an answering machine detection algorithm, a voice activity detection algorithm, and a call progress analysis algorithm.

### Claim 10 (depends on 6)

The method of claim 6, wherein step (a) comprises transforming the candidate audio segment using a transform selected from the group consisting of: discrete fourier transform, discrete cosine transform, wavelet transform, and filter bank.

### Claim 11 (depends on 7)

The method of claim 7, wherein step (b) comprises the steps of:
b.1) transforming the spectrogram data into Mel space with N+1 frames, where N is an integer;
b.2) computing N-bit binary fingerprint frame values based on band energy differences over successive frames according to the following formula:
F

(
n
,
m
)
=
{
1

if


E

(
n
,
m
)
-
E

(
n
,
m
+
1
)
-
(
E

(
n
-
1
,
m
)
-
E

(
n
-
1
,
m
+
1
)
)
>
0
0

if


E

(
n
,
m
)
-
E

(
n
,
m
+
1
)
-
(
E

(
n
-
1
,
m
)
-
E

(
n
-
1
,
m
+
1
)
)
≤
0
where: E(n,m) is an energy of frequency band m of frame n, and
F(n,m) is an mth bit of fingerprint frame n.

### Claim 12 (depends on 6)

The method of claim 6, wherein step (c) comprises computing a Hamming distance between the candidate binary fingerprint bitmap and the at least one known binary fingerprint bitmap.

### Claim 13 (depends on 6)

The method of claim 6, wherein the predetermined threshold is a bit error rate.

### Claim 14

A method for creating a ternary bitmap of a dataset, the method comprising the steps of:
a) computing a binary fingerprint bitmap of the dataset;
b) deleting a first number of samples from the dataset;
c) after step (b), computing another binary fingerprint bitmap of the dataset;
d) repeating steps (b) and (c) a plurality of times to create a plurality of binary fingerprint bitmaps; and
e) combining the plurality of binary fingerprint bitmaps into a ternary bitmap, where each bit in the ternary bitmap is determined as follows:
e.1) If a bit is 0 (zero) in a first predetermined number of the plurality of binary bitmaps, set the bit in the ternary bitmap to 0 (zero);
e.2) If a bit is 1 (one) in a second predetermined number of the plurality of binary bitmaps, set the bit in the ternary bitmap to 1 (one); and
e.3) Otherwise, set the bit of the ternary bitmap to * (“Don't Care”).

### Claim 15 (depends on 14)

The method of claim 14, wherein said first number of samples comprises a first sample from the dataset.

### Claim 16 (depends on 14)

The method of claim 14, wherein said first predetermined number comprises all of the plurality of binary bitmaps and said second predetermined number comprises all of the plurality of binary bitmaps.

### Claim 17 (depends on 14)

The method of claim 14, wherein said first predetermined number and said second predetermined number are equal.

### Claim 18 (depends on 14)

The method of claim 14, wherein the dataset is artificially degraded prior to step (a).

### Claim 19 (depends on 14)

The method of claim 14, wherein said binary fingerprint bitmap comprises an acoustic fingerprint bitmap.

### Claim 20 (depends on 14)

The method of claim 14, wherein said dataset comprises an audio segment.

### Claim 21 (depends on 20)

The method of claim 20, wherein said audio segment comprises an audio segment from an outbound telephone call.

### Claim 22

A method for identifying a candidate dataset, the method comprising the steps of:
a) computing a binary fingerprint bitmap of a known dataset in a known dataset database;
b) deleting a first number of samples from the known dataset;
c) after step (b), computing another binary fingerprint bitmap of the known dataset;
d) repeating steps (b) and (c) a plurality of times to create a plurality of binary fingerprint bitmaps; and
e) combining the plurality of binary fingerprint bitmaps into a ternary bitmap, where each bit in the ternary bitmap is determined as follows:
e.1) If a bit is 0 in a first predetermined number of the plurality of binary bitmaps, set the bit in the ternary bitmap to 0;
e.2) If a bit is 1 in a second predetermined number of the plurality of binary bitmaps, set the bit in the ternary bitmap to 1; and
e.3) Otherwise, set the bit of the ternary bitmap to * (“Don't Care”);
f) saving the ternary bitmap into a ternary bitmap database;
g) repeating steps (a)-(f) for all known datasets in the known dataset database;
h) creating a candidate dataset binary fingerprint bitmap from the candidate dataset; and
i) comparing the candidate dataset binary fingerprint bitmap to each ternary bitmap in the ternary bitmap database, wherein said comparison ignores the Don't Care bits.

### Claim 23 (depends on 22)

The method of claim 22, wherein said first number of samples comprises a first sample from the dataset.

### Claim 24 (depends on 22)

The method of claim 22, wherein said first predetermined number comprises all of the plurality of binary bitmaps and said second predetermined number comprises all of the plurality of binary bitmaps.

### Claim 25 (depends on 22)

The method of claim 22, wherein said first predetermined number and said second predetermined number are equal.

### Claim 26 (depends on 22)

The method of claim 22, wherein the dataset is artificially degraded prior to step (a).

### Claim 27 (depends on 22)

The method of claim 22, wherein said binary fingerprint bitmap and said candidate binary fingerprint bitmap each comprise an acoustic fingerprint bitmap.

### Claim 28 (depends on 22)

The method of claim 22, wherein said dataset comprises an audio segment.

### Claim 29 (depends on 28)

The method of claim 28, wherein said audio segment comprises an audio segment from an outbound telephone call.

### Claim 30

A method for creating a ternary bitmap of an audio segment, the method comprising the steps of:
a) computing a binary acoustic fingerprint bitmap of the audio segment;
b) deleting a first number of samples from the audio segment;
c) after step (b), computing another binary acoustic fingerprint bitmap of the audio segment;
d) repeating steps (b) and (c) a plurality of times to create a plurality of binary acoustic fingerprint bitmaps; and
e) combining the plurality of binary acoustic fingerprint bitmaps into a ternary bitmap, where each bit in the ternary bitmap is determined as follows:
e.1) If a bit is 0 in a first predetermined number of the plurality of binary bitmaps, set the bit in the ternary bitmap to 0;
e.2) If a bit is 1 in a second predetermined number of the plurality of binary bitmaps, set the bit in the ternary bitmap to 1; and
e.3) Otherwise, set the bit of the ternary bitmap to * (“Don't Care”).

### Claim 31 (depends on 30)

The method of claim 30, wherein said first number of samples comprises a first sample from the audio segment.

### Claim 32 (depends on 30)

The method of claim 30, wherein said first predetermined number comprises all of the plurality of binary bitmaps and said second predetermined number comprises all of the plurality of binary bitmaps.

### Claim 33 (depends on 30)

The method of claim 30, wherein said first predetermined number and said second predetermined number are equal.

### Claim 34 (depends on 30)

The method of claim 30, wherein the dataset is artificially degraded prior to step (a).

### Claim 35 (depends on 30)

The method of claim 30, wherein said audio segment comprises an audio segment from an outbound telephone call.

### Claim 36

A method for identifying a candidate audio segment, the method comprising the steps of:
a) computing a binary acoustic fingerprint bitmap of a known audio segment in a known audio segment database;
b) deleting a first number of samples from the known audio segment;
c) after step (b), computing another binary acoustic fingerprint bitmap of the known audio segment;
d) repeating steps (b) and (c) a plurality of times to create a plurality of binary acoustic fingerprint bitmaps; and
e) combining the plurality of binary acoustic fingerprint bitmaps into a ternary bitmap, where each bit in the ternary bitmap is determined as follows:
e. 1) If a bit is 0 in a first predetermined number of the plurality of binary bitmaps, set the bit in the ternary bitmap to 0;
e.2) If a bit is 1 in a second predetermined number of the plurality of binary bitmaps, set the bit in the ternary bitmap to 1; and
e.3) Otherwise, set the bit of the ternary bitmap to * (“Don't Care”);
f) saving the ternary bitmap into a ternary bitmap database;
g) repeating steps (a)-(f) for all known audio segments in the known audio segment database;
h) creating a candidate audio segment binary acoustic fingerprint bitmap from the candidate audio segment; and
i) comparing the candidate audio segment binary acoustic fingerprint bitmap to each ternary bitmap in the ternary bitmap database, wherein said comparison ignores the Don't Care bits.

### Claim 37 (depends on 36)

The method of claim 36, wherein said first number of samples comprises a first sample from the dataset.

### Claim 38 (depends on 36)

The method of claim 36, wherein said first predetermined number comprises all of the plurality of binary bitmaps and said second predetermined number comprises all of the plurality of binary bitmaps.

### Claim 39 (depends on 36)

The method of claim 36, wherein said first predetermined number and said second predetermined number are equal.

### Claim 40 (depends on 36)

The method of claim 36, wherein the dataset is artificially degraded prior to step (a).

### Claim 41 (depends on 36)

The method of claim 36, wherein said audio segment comprises an audio segment from an outbound telephone call.

### Claim 42

A method for creating a weighted compressed representation of a dataset, the method comprising the steps of:
a) computing a compressed representation of the dataset;
b) applying a transformation to the dataset;
c) after step (b), computing another compressed representation of the dataset;
d) repeating steps (b) and (c) a plurality of times to create a plurality of compressed representations; and
e) combining the plurality of compressed representations into a weighted compressed representation, where each weight in the weighted compressed representation is computed as a function of elements in the plurality of compressed representations.

### Claim 43 (depends on 42)

The method of claim 42, wherein said compressed representation is a two-dimensional vector of first numbers.

### Claim 44 (depends on 43)

The method of claim 43, wherein said first numbers are each represented with a first number of bits.

### Claim 45 (depends on 44)

The method of claim 44, wherein said first number of bits is one.

### Claim 46 (depends on 42)

The method of claim 42, wherein said weighted compressed representation is a two-dimensional vector of second numbers.

### Claim 47 (depends on 46)

The method of claim 46, wherein said second numbers are represented with a second number of bits each.

### Claim 48 (depends on 42)

The method of claim 42, wherein said function computes said each weight as correlation coefficient of the elements in said plurality of compressed representations.

### Claim 49 (depends on 48)

The method of claim 48, wherein said correlation coefficient is computed as follows:
1) if an element is below a first predetermined threshold in a first predetermined number of said plurality of compressed representations, set the correlation coefficient to a first value;
2) if an element is above a second predetermined threshold in a second predetermined number of said plurality of compressed representations, set the correlation coefficient to a second value;
3) otherwise set the correlation coefficient to a third value.

### Claim 50 (depends on 49)

The method of claim 49, wherein said first value is +1, said second value is −1, and said third value is 0 (zero).

### Claim 51 (depends on 49)

The method of claim 49, wherein said first predetermined threshold is 1 (one) and said second predetermined threshold is 0 (zero).

### Claim 52 (depends on 49)

The method of claim 49, wherein said first predetermined number comprises all of the plurality of compressed representations and said second predetermined number comprises all of the plurality of compressed representations.

### Claim 53 (depends on 42)

The method of claim 42, wherein said transformation comprises deleting a first number of elements from said dataset.

### Claim 54 (depends on 42)

The method of claim 42, wherein said first number is one.
