# US 2025/0095669 A1 · Fake voice detection method based on dual-track differential modeling

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0095669 A1
- Document ID: US20250095669A1
- Status: Published application
- Kind: A1
- Application number: 18/537,818
- Filed: 2023-12-13
- Published: 2025-03-20
- Earliest claimed: 2023-09-20
- Assignees: Inner Mongolia University
- Applicants: Inner Mongolia University
- Inventors: Rui LIU; Jinhua ZHANG; Yifan HU; Haolin ZUO; Yuxuan MA
- Web page: https://patentgrep.com/patent/US20250095669A1

## CPC

- G10L 17/26: Recognition of special voice characteristics, e.g. for use in lie detectors; Recognition of animal voices
- G10L 25/51: for comparison or discrimination
- G10L 17/04: Training, enrolment or model building
- G10L 25/18: the extracted parameters being spectral information of each sub-band
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

A fake voice detection method based on dual-track differential modeling includes: converting, by a pre-trained single/dual-track voice conversion model, a single-track audio into a stereo; extracting left and right-track Mel spectrogram features; performing texture enhancement on absolute values of differences between an original single-track Mel spectrogram feature and each of the left and right-track Mel spectrogram features; inputting texture enhancement results into left and right dual-branch feature extractors respectively to acquire a final attention map through feature extraction, processing and fusion; and inputting the final attention map into an attention pooling layer and a final binary classification layer to acquire a fake voice detection result. The fake voice detection method achieves fake voice detection by means of dual-track differential modeling, fine-grained texture enhancement, and multi-head attention feature fusion, improving the accuracy, transferability, and generalization of fake voice detection.

## Claims

### Claim 1

A fake voice detection method based on dual-track differential modeling, comprising the following steps:
S1: converting, by a pre-trained single/dual-track voice conversion model, a single-track audio to a stereo; and extracting a left-track Mel spectrogram feature Y1:TL and a right-track Mel spectrogram feature Y1:TR;
S2: acquiring an absolute value Y1:TL′ of a difference between the left-track Mel spectrogram feature Y1:TL and an original single-track Mel spectrogram feature and an absolute value Y1:TR′ of a difference between the right-track Mel spectrogram feature Y1:TR and the original single-track Mel spectrogram feature;
S3: performing texture enhancement on the absolute values Y1:TL′ and Y1:TR′; extracting and fusing, by two dual-branch feature extractors, texture enhancement results to acquire a final attention map G′L/R, and
S4: acquiring, by an attention pooling layer and a binary classification layer, a fake voice detection result based on the final attention map G′L/R.

### Claim 2 (depends on 1)

The fake voice detection method based on dual-track differential modeling according to claim 1, wherein step S1 comprises pre-training as follows:
A1: acquiring a paired audio dataset as a training dataset, wherein the paired audio dataset comprises a conditional time signal C1:T representing positions and directions of a source and a listener;
A2: reading, by a neural time warping module, the conditional time signal C1:T, and predicting a neural warping field ρ; and modifying and simulating, by a time convolutional module, the predicted neural warping field ρ to acquire an optimal neural warping field ρ′;
A3: calculating, by a recursive activation function, a left-ear curling signal X1:TL and a right-ear curling signal X1:TR based on the neural warping field ρ and the optimal neural warping field ρ′;
A4: reconstructing, by a time convolutional network comprising a wave network, the left-track Mel spectrogram feature Y1:TL and the right-track Mel spectrogram feature Y1:TR based on the left-ear curling signal X1:TL, the right-ear curling signal X1:TR, and the conditional time signal C1:T;
A5: repeating steps A1 to A4 for a total of N times to acquire parameters, left-track Mel spectrogram features Y1:TL, and right-track Mel spectrogram features Y1:TR of N single/dual-track voice conversion models with different training times; and
A6: comparing the left-track Mel spectrogram features Y1:TL and the right-track Mel spectrogram features Y1:TR of the N single/dual-track voice conversion models with different training times, and selecting parameters of a single/dual-track voice conversion model with an optimal effect in the Mel spectrogram feature as parameters of a final pre-trained model.

### Claim 3 (depends on 1)

The fake voice detection method based on dual-track differential modeling according to claim 1, wherein in step S3, each of the two dual-branch feature extractors comprises a convolutional filter layer, a residual network layer, a graph attention network layer, and a graph pooling layer, wherein
the convolutional filter layer and the residual network layer are configured to perform feature extraction on the absolute values Y1:TL′ and Y1:TR′ to acquire a left-track feature h′1:TL and a right-track feature h′1:TR;
the graph attention network layer is configured to perform attention weight calculation on a spatial attention map G to acquire an aggregation result of adjacent nodes; and
the graph pooling layer is configured to select a subset of nodes with a highest amount of information from the aggregation result of the adjacent nodes, form a new attention map, and acquire the final attention map G′L/R.

### Claim 4 (depends on 1)

The fake voice detection method based on dual-track differential modeling according to claim 1, wherein step S3 comprises:
S31: performing texture enhancement on the absolute values Y1:TL′ and Y1:TR′ ; and inputting the texture enhancement results into the two dual-branch feature extractors, respectively;
S32: performing, by a convolutional filter layer and a residual network layer, feature extraction on the absolute values Y1:TL′ and Y1:TR′, respectively, to acquire a left-track feature h′1:TL and a right-track feature h′1:TR;
S33: predicting, by a multi-head attention module, spatial attention maps based on the left-track feature h′1:TL and the right-track feature h′1:TR to acquire a left-track spatial attention map h′1:TL′ and a right-track spatial attention map h′1:TR′;
S34: performing, by an attention module, feature information fusion on the left-track spatial attention map h′1:TL′ and the right-track spatial attention map h′1:TR′ to acquire a spatial attention map G;
S35: calculating, by a graph attention network layer, an attention weight of the spatial attention map G, and aggregating adjacent nodes to acquire an aggregation result; and
S36: selecting, by a graph pooling layer, a subset of nodes with a highest amount of information from the aggregation result, forming a new attention map, and acquiring the final attention map G′L/R.

### Claim 5 (depends on 4)

The fake voice detection method based on dual-track differential modeling according to claim 4, wherein in step S31, the texture enhancement comprises:
B1: downsampling, by a local average pooling layer, feature maps of the absolute values Y1:TL′ and Y1:TR′ to acquire a pooled feature map;
B2: performing, by a residual module, residual processing on the pooled feature map to acquire track texture information; and
B3: enhancing, by a convolutional block, the track texture information to acquire the left-track Mel spectrogram feature and the right-track Mel spectrogram feature after the texture enhancement.

### Claim 6 (depends on 5)

The fake voice detection method based on dual-track differential modeling according to claim 5, wherein in step B2, the residual processing is expressed as follows:
T
L
/
R
=
f
⁡
(
a
L
/
R
)
-
D
L
/
R
wherein TL/R denotes the track texture information generated through the residual processing; L and R respectively denote absolute values of a left-track Mel spectrogram feature difference and a right-track Mel spectrogram feature difference of the stereo; f(aL/R) denotes a feature map of the absolute values of the left-track Mel spectrogram feature difference and the right-track Mel spectrogram feature difference; and DL/R denotes the pooled feature map.

### Claim 7 (depends on 4)

The fake voice detection method based on dual-track differential modeling according to claim 4, wherein in step S33, the left-track spatial attention map h′1:TL′ and the right-track spatial attention map h′1:TR′ are expressed as follows:
h′1:TL=fL(Y1:TL′)
h′1:TR=fR(Y1:TR′)
wherein Y1:TL′ denotes the absolute value of the difference between the left-track Mel spectrogram feature and the original single-track Mel spectrogram feature; Y1:TR′ denotes the absolute value of the difference between the right-track Mel spectrogram feature and the original single-track Mel spectrogram feature; fL(.) and fR(.) respectively denote an extraction process function of the left-track Mel spectrogram feature and an extraction process function of the right-track Mel spectrogram feature; and h′1:TL and h′1:TR respectively denote the left-track feature and the right-track feature.

### Claim 8 (depends on 4)

The fake voice detection method based on dual-track differential modeling according to claim 4, wherein in step S34, the spatial attention map G is expressed as follows:
G=G(N, ε, h′)
wherein N denotes a number of nodes in the spatial attention map; ε denotes connecting edges between all nodes, comprising a self-connection; and h′ denotes feature representation.

### Claim 9 (depends on 4)

The fake voice detection method based on dual-track differential modeling according to claim 4, wherein in step S35, the attention weight is expressed as follows:
α
u
,
n
=
exp
⁡
(
W
(
h
n
′
⊙
h
u
′
)
)
∑
w
⁢
ϵ
⁢
M
⁡
(
n
)
⋃
{
n
}
exp
⁡
(
W
⁡
(
h
n
′
⊙
h
w
′
)
)
wherein u and n denote nodes; αu,n denotes an attention weight after the node u and the node n are aggregated; exp(.) denotes an exponential function; M(n) denotes an adjacent node of the node n; W denotes a learnable weight; h′n denotes a feature vector of the node n; h′u denotes a feature vector of the node u; ⊙ denotes multiplication at an element level; and h′w denotes a feature vector of a node w.

### Claim 10 (depends on 4)

The fake voice detection method based on dual-track differential modeling according to claim 4, wherein in step S36, information of the n nodes in the final attention map G′L/R is expressed as follows:
o
n
=
ReLU
⁡
(
BN
⁡
(
m
n
+
h
n
′
)
)
m
m
=
∑
u
∈
M
⁡
(
n
)
⋃
{
n
}
α
u
,
n
⁢
h
u
′
wherein on denotes the information of the n nodes in the final attention map G′L/R; mn denotes information after the n nodes are aggregated; ReLU denotes an activation function; BN denotes batch normalization; mi denotes aggregation information of an i-th node; h′n denotes a feature vector of a node n; M(n) denotes a set of adjacent nodes of the node n; αu,n denotes an attention weight between a node u and the node n; and h′u denotes a feature vector of the node u.
