# US 12,592,239 B2 · Active voice liveness detection system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,592,239 B2
- Document ID: US12592239B2
- Status: Granted
- Kind: B2
- Application number: 18/646,310
- Filed: 2024-04-25
- Published: 2024-10-31
- Granted: 2026-03-31
- Earliest claimed: 2023-04-28
- Assignees: Pindrop Security, Inc.
- Applicants: Pindrop Security, Inc.
- Inventors: Elie Khoury; Ganesh Sivaraman; Tianxiang Chen; Nikolay Gaubitch; David Looney; Amit Gupta; Vijay Balasubramaniyan; Nicholas Klein; Anthony Stankus
- Web page: https://patentgrep.com/patent/US12592239B2

## CPC

- G10L 17/26: Recognition of special voice characteristics, e.g. for use in lie detectors; Recognition of animal voices
- G06F 21/32: using biometric data, e.g. fingerprints, iris scans or voiceprints
- G10L 15/02: Feature extraction for speech recognition; Selection of recognition unit
- G10L 17/00: Speaker identification or verification techniques
- G10L 17/02: Preprocessing operations, e.g. segment selection; Pattern representation or modelling, e.g. based on linear discriminant analysis [LDA] or principal components; Feature selection or extraction
- G10L 17/04: Training, enrolment or model building
- G10L 17/08: Use of distortion metrics or a particular distance between probe pattern and reference templates
- G10L 17/18: Artificial neural networks; Connectionist approaches
- G10L 25/51: for comparison or discrimination
- G10L 25/60: for measuring the quality of voice signals
- H04M 3/2281: {Call monitoring, e.g. for law enforcement purposes; Call tracing; Detection or prevention of malicious calls}
- H04M 3/42221: {Conversation recording systems (at the subscriber's set H04M1/656)}
- H04M 3/5175: {Call or contact centers supervision arrangements}
- H04M 3/5183: {Call or contact centers with computer-telephony arrangements}
- G10L 25/30: using neural networks
- H04M 2201/405: involving speaker-dependent recognition
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04M: TELEPHONIC COMMUNICATION (circuits for controlling other apparatus via a telephone cable and not involving telephone switching apparatus G08)

## Abstract

Disclosed are systems and methods including software processes executed by a server that detect audio-based synthetic speech (“deepfakes”) in a call conversation. Embodiments include systems and methods for detecting fraudulent presentation attacks using multiple functional engines that implement various fraud-detection techniques, to produce calibrated scores and/or fused scores. A computer may, for example, evaluate the audio quality of speech signals within audio signals, where speech signals contain the speech portions having speaker utterances.

## Claims

### Claim 1

A computer-implemented method for detecting machine-based speech in calls, comprising:
obtaining, by a computer, an inbound audio signal comprising a speech signal containing response content as an utterance of the speaker, wherein the response content in the speech signal purportedly matches to challenge content of a verification prompt;
extracting, by the computer, a text embedding using a first set of features extracted for text of the challenge content, a spoken content embedding using a second set of features extracted for the speech signal, and a fakeprint using a third set one or more features extracted for one or more fraud artifacts of the speech signal;
generating, by the computer, a content verification score based upon a distance between the text embedding and the spoken content embedding;
executing, by the computer, a passive liveness detector to generate a passive liveness score for the inbound audio signal, the passive liveness detector having a set of layers of a machine-learning architecture trained to classify and score the input audio signal based upon the fakeprint extracted for the fraud artifacts of the inbound audio signal;
generating, by the computer, a fused liveness score based upon the content verification score and the passive liveness score; and
identifying, by the computer, the inbound audio signal as genuine or fraudulent based upon comparing the fused liveness score against an overall risk threshold.

### Claim 2 (depends on 1)

The method according to claim 1, further comprising:
extracting, by the computer, an inbound voiceprint for the speech signal using a fourth set of one or more features extracted for one or more acoustic features of the speech signal of the inbound audio signal; and
generating, by the computer, a speaker verification score for the speech signal indicating a speaker recognition likelihood that the speaker is an enrolled user based upon a second distance between the inbound voiceprint and an enrolled voiceprint.

### Claim 3 (depends on 2)

The method according to claim 2, wherein the computer generates the fused liveness score further using the speaker verification score.

### Claim 4 (depends on 1)

The method according to claim 1, further comprising:
generating, by the computer, one or more acoustic parameters corresponding to one or more types of degradation in the speech signal of the inbound audio signal; and
generating, by the computer, a speech quality score for the speech signal based upon the one or more acoustic parameters.

### Claim 5 (depends on 4)

The method according to claim 4, wherein generating the content verification score includes:
calibrating, by the computer, the content verification score based upon the speech quality score.

### Claim 6 (depends on 4)

The method according to claim 4, further comprising:
determining, by the computer, that the speech quality score for the speech signal fails a speech quality threshold; and
transmitting, by the computer, to the user device a request for an improved speech signal for the caller.

### Claim 7 (depends on 1)

The method according to claim 1, further comprising:
extracting, by the computer, an inbound audioprint using one or more features extracted from the audio signal;
generating, by the computer, an audio replay score for the inbound audio signal indicating an audio recording recognition likelihood that the inbound audio signal matches a prior audio signal based upon a distance between the inbound audioprint and a stored audioprint for the prior audio signal.

### Claim 8 (depends on 7)

The method according to claim 7, further comprising identifying, by the computer, the inbound audio signal as fraudulent, in response to determining that the audio replay score satisfies a replay detection threshold value.

### Claim 9 (depends on 7)

The method according to claim 7, further comprising storing, by the computer, the inbound audioprint into a database as a new stored audioprint.

### Claim 10 (depends on 1)

The method according to claim 1, further comprising generating, by the computer, a verification prompt including the challenge content for display at a user interface of the user device associated with the caller.

### Claim 11

A system for detecting machine-based speech in calls, comprising:
a computer having at least one processor, configured to:
obtain an inbound audio signal comprising a speech signal containing response content as an utterance of the speaker, wherein the response content in the speech signal purportedly matches to challenge content of a verification prompt;
extract a text embedding using a first set of features extracted for text of the challenge content, a spoken content embedding using a second set of features extracted for the speech signal, and a fakeprint using a third set one or more features extracted for one or more fraud artifacts of the speech signal;
generate a content verification score based upon a distance between the text embedding and the spoken content embedding;
execute a passive liveness detector having a set of layers of a machine-learning architecture to generate a passive liveness score for the inbound audio signal, the passive liveness detector trained to classify and score the input audio signal based upon the fakeprint extracted for the fraud artifacts of the inbound audio signal;
generate a fused liveness score based upon the content verification score and the passive liveness score; and
identify the inbound audio signal as genuine or fraudulent based upon comparing the fused liveness score against an overall risk threshold.

### Claim 12 (depends on 11)

The system according to claim 11, wherein the computer is further configured to:
extract an inbound voiceprint for the speech signal using a fourth set of one or more features extracted for one or more acoustic features of the speech signal of the inbound audio signal; and
generate a speaker verification score for the speech signal indicating a speaker recognition likelihood that the speaker is an enrolled user based upon a second distance between the inbound voiceprint and an enrolled voiceprint.

### Claim 13 (depends on 12)

The system according to claim 12, wherein the computer generates the fused liveness score further using the speaker verification score.

### Claim 14 (depends on 11)

The system according to claim 11, wherein the computer is further configured to:
generate one or more acoustic parameters corresponding to one or more types of degradation in the speech signal of the inbound audio signal; and
generate a speech quality score for the speech signal based upon the one or more acoustic parameters.

### Claim 15 (depends on 14)

The system according to claim 14, wherein when generating the content verification score the computer is further configured to calibrate the content verification score based upon the speech quality score.

### Claim 16 (depends on 14)

The system according to claim 14, wherein the computer is further configured to:
determine that the speech quality score for the speech signal fails a speech quality threshold; and
transmit to the user device a request for an improved speech signal for the caller.

### Claim 17 (depends on 11)

The system according to claim 11, wherein the computer is further configured to:
extract an inbound audioprint using one or more features extracted from the audio signal;
generate an audio replay score for the inbound audio signal indicating an audio recording recognition likelihood that the inbound audio signal matches a prior audio signal based upon a distance between the inbound audioprint and a stored audioprint for the prior audio signal.

### Claim 18 (depends on 17)

The system according to claim 17, wherein the computer is further configured to identify the inbound audio signal as fraudulent, in response to determining that the audio replay score satisfies a replay detection threshold value.

### Claim 19 (depends on 17)

The system according to claim 17, wherein the computer is further configured to store the inbound audioprint into a database as a new stored audioprint.

### Claim 20 (depends on 11)

The system according to claim 11, wherein the computer is further configured to generate a verification prompt including the challenge content for display at a user interface of the user device associated with the caller.