# US 12,002,484 B2 · Method and apparatus for post-processing audio signal, storage medium, and electronic device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,002,484 B2
- Document ID: US12002484B2
- Status: Granted
- Kind: B2
- Application number: 17/736,797
- Filed: 2022-05-04
- Published: 2022-08-18
- Granted: 2024-06-04
- Earliest claimed: 2020-05-14
- Assignees: Tencent Technology (Shenzhen) Company Limited
- Applicants: Tencent Technology (Shenzhen) Company Limited
- Inventors: Yang Yu; Yu Chen
- Web page: https://patentgrep.com/patent/US12002484B2

## CPC

- G10L 21/0232: Processing in the frequency domain
- G10L 21/06: Transformation of speech into a non-audible representation, e.g. speech visualisation or speech processing for tactile aids (G10L15/26 takes precedence)
- G06F 21/6245: {Protecting personal data, e.g. for financial or medical purposes}
- G10L 21/0208: Noise filtering
- G10L 21/0224: Processing in the time domain
- H04K 1/04: by frequency scrambling, i.e. by transposing or inverting parts of the frequency band or by inverting the whole band
- H04R 3/002: {Damping circuit arrangements for transducers, e.g. motional feedback circuits}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04K: SECRET COMMUNICATION; JAMMING OF COMMUNICATION
- H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K)

## Abstract

This application discloses a method and an apparatus for processing an audio signal. The method includes obtaining a first speech signal acquired by a first device; performing frame blocking on the first speech signal, to obtain multiple speech signal frames; converting the multiple speech signal frames into multiple first frequency domain signal frames; performing aliasing processing on a first sub-frequency domain signal frame among the multiple first frequency domain signal frames with a frequency lower than or equal to a target frequency threshold, and retaining a second sub-frequency domain signal frame among the multiple first frequency domain signal frames with a frequency higher than the target frequency threshold, to obtain multiple second frequency domain signal frames, the target frequency threshold being related to a sampling frequency of a second device; and performing frame fusion on the multiple second frequency domain signal frames, to obtain a second speech signal.

## Claims

### Claim 1

A method for processing an audio signal, performed by an electronic device, the method comprising:
obtaining a first speech signal acquired by a first device;
performing frame blocking on the first speech signal, to obtain multiple speech signal frames;
converting the multiple speech signal frames into multiple first frequency domain signal frames;
performing aliasing processing on a first sub-frequency domain signal frame with a frequency lower than or equal to a target frequency threshold in each first frequency domain signal frame in the multiple first frequency domain signal frames, and reserving a second sub-frequency domain signal frame among the multiple first frequency domain signal frames with a frequency higher than the target frequency threshold in the each first frequency domain signal frame, to obtain multiple second frequency domain signal frames, the target frequency threshold being related to a sampling frequency of a second device, comprising:
alternately enhancing and attenuating the first sub-frequency domain signal in the each first frequency domain signal frame, to obtain a third sub-frequency domain signal frame, and reserving the second sub-frequency domain signal in the each first frequency domain signal frame, to obtain the multiple second frequency domain signal frames, wherein a kth second frequency domain signal frame Xw′[k,n] of the multiple second frequency domain signal frames equals to the third sub-frequency domain signal frame obtained after aliasing processing is performed on the first sub-frequency domain signal frame in a kth first frequency domain signal frame of the first frequency domain signal frames when a frequency value n is less than or equal to the target frequency threshold; and
performing frame fusion on the multiple second frequency domain signal frames, to obtain a second speech signal.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the multiple second frequency domain signal frames satisfy the following relationship:
X
w
′
[
k
,
n
]
=
{
X
w
[
k
,
n
]
⁢
H
[
k
,
n
]
,
n
≤
f
th
X
w
[
k
,
n
]
,
n
>
f
th
wherein H[k,n] is a low-frequency aliasing function of a kth frame, fth represents the target frequency threshold, and Xw[k,n] represents the kth first frequency domain signal frame; when n≤fth, Xw[k,n] represents a first sub-frequency domain signal frame in the kth first frequency domain signal frame; when n>fth, Xw[k,n] represents a second sub-frequency domain signal frame in the kth first frequency domain signal frame, Xw[k,n]H[k,n] represents the third sub-frequency domain signal frame obtained after the aliasing processing is performed on the first sub-frequency domain signal frame in the kth first frequency domain signal frame, and Xw′[k,n] represents the kth second frequency domain signal frame.

### Claim 3 (depends on 1)

The method according to claim 1, wherein the multiple second frequency domain signal frames satisfy the following relationship:
X
w
′
[
k
,
n
]
=
{
X
w
[
k
,
n
]
+
H
[
k
,
n
]
,
n
≤
f
th
X
w
[
k
,
n
]
,
n
>
f
th
wherein H[k,n] is a low-frequency aliasing function of a kth frame, fth represents the target frequency threshold, and Xw[k,n] represents a kth first frequency domain signal frame; when n≤fth, Xw[k,n] represents a first sub-frequency domain signal frame in the kth first frequency domain signal frame; when n>fth, Xw[k,n] represents a second sub-frequency domain signal frame in the kth first frequency domain signal frame, Xw[k,n]+H[k,n] represents the third sub-frequency domain signal frame obtained after aliasing processing is performed on the first sub-frequency domain signal frame in the kth first frequency domain signal frame, and Xw′[k,n] represents the kth second frequency domain signal frame.

### Claim 4 (depends on 1)

The method according to claim 1, wherein
the performing frame blocking on the first speech signal, to obtain multiple speech signal frames comprises:
dividing the first speech signal into the multiple speech signal frames according to frame duration determined based on a Haas effect and a preset sampling frequency;
the converting the multiple speech signal frames into multiple first frequency domain signal frames comprises:
adding an analysis window ha to each speech signal frame in the multiple speech signal frames to obtain multiple first windowed signals; and
performing discrete Fourier transform on each first windowed signal frame in the multiple first windowed signals, to obtain the multiple first frequency domain signal frames;
the performing frame fusion on the multiple second frequency domain signal frames, to obtain a second speech signal comprises:
performing inverse discrete Fourier transform on each second frequency domain signal frame in the multiple second frequency domain signal frames, to obtain multiple first time domain signal frames;
adding a synthesis window hs to each first time domain signal frame in the multiple first time domain signal frames, to obtain multiple second windowed signal frames; and
performing frame fusion on the multiple second windowed signal frames, to obtain the second speech signal.

### Claim 5 (depends on 4)

The method according to claim 4, wherein a frame length N of each speech signal frame in the multiple speech signal frames, the analysis window ha and the synthesis window hs satisfy the following relationship:
∑
n
=
0
N
-
1
h
a
(
n
)
⁢
h
s
(
n
)
=
1
wherein ha(n) represents an nth item in the analysis window ha, and hs(n) represents an nth item in the synthesis window hs.

### Claim 6 (depends on 1)

The method according to claim 1, wherein a frame length of each speech signal frame in the multiple speech signal frames is N, the frame length N satisfy the following relationship:
N≈α×f
wherein f is the sampling frequency of the second device, and α is frame duration of each speech signal frame in the multiple speech signal frames.

### Claim 7 (depends on 1)

The method according to claim 1, wherein the target frequency threshold being related to the sampling frequency of the second device comprises:
the target frequency threshold being the same as an upper limit of the sampling frequency of the second device.

### Claim 8

An apparatus for processing an audio signal, comprising a memory and a processor coupled to the memory, the processor being configured to:
obtain a first speech signal acquired by a first device;
perform frame blocking on the first speech signal, to obtain multiple speech signal frames;
convert the multiple speech signal frames into multiple first frequency domain signal frames;
perform aliasing processing on a first sub-frequency domain signal frame with a frequency lower than or equal to a target frequency threshold in each first frequency domain signal frame in the multiple first frequency domain signal frames, and reserve a second sub-frequency domain signal frame among the multiple first frequency domain signal frames with a frequency higher than the target frequency threshold in the each first frequency domain signal frames, to obtain multiple second frequency domain signal frames, the target frequency threshold being related to a sampling frequency of a second device, comprising:
alternately enhancing and attenuating the first sub-frequency domain signal in the each first frequency domain signal frame, to obtain a third sub-frequency domain signal frame, and reserving the second sub-frequency domain signal in the each first frequency domain signal frame, to obtain the multiple second frequency domain signal frames, wherein a kth second frequency domain signal frame Xw′[k,n] of the multiple second frequency domain signal frames equals to the third sub-frequency domain signal frame obtained after aliasing processing is performed on the first sub-frequency domain signal frame in a kth first frequency domain signal frame of the first frequency domain signal frames when a frequency value n is less than or equal to the target frequency threshold; and
perform frame fusion on the multiple second frequency domain signal frames, to obtain a second speech signal.

### Claim 9 (depends on 8)

The apparatus according to claim 8, wherein the multiple second frequency domain signal frames satisfy the following relationship:
X
w
′
[
k
,
n
]
=
{
X
w
[
k
,
n
]
⁢
H
[
k
,
n
]
,
n
≤
f
th
X
w
[
k
,
n
]
,
n
>
f
th
wherein H[k,n] is a low-frequency aliasing function of a kth frame, fth represents the target frequency threshold, and Xw[k,n] represents the kth first frequency domain signal frame; when n≤fth, Xw[k,n] represents a first sub-frequency domain signal frame in the kth first frequency domain signal frame; when n>fth, Xw[k,n] represents a second sub-frequency domain signal frame in the kth first frequency domain signal frame, Xw[k,n]H[k,n] represents the third sub-frequency domain signal frame obtained after the aliasing processing is performed on the first sub-frequency domain signal frame in the kth first frequency domain signal frame, and Xw′[k,n] represents the kth second frequency domain signal frame.

### Claim 10 (depends on 8)

The apparatus according to claim 8, wherein the multiple second frequency domain signal frames satisfy the following relationship:
X
w
′
[
k
,
n
]
=
{
X
w
[
k
,
n
]
+
H
[
k
,
n
]
,
n
≤
f
th
X
w
[
k
,
n
]
,
n
>
f
th
wherein H[k,n] is a low-frequency aliasing function of a kth frame, fth represents the target frequency threshold, and Xw[k,n] represents a kth first frequency domain signal frame; when n≤fth, Xw[k,n] represents a first sub-frequency domain signal frame in the kth first frequency domain signal frame; when n>fth, Xw[k,n] represents a second sub-frequency domain signal frame in the kth first frequency domain signal frame, Xw[k,n]+H[k,n] represents the third sub-frequency domain signal frame obtained after aliasing processing is performed on the first sub-frequency domain signal frame in the kth first frequency domain signal frame, and Xw′[k,n] represents the kth second frequency domain signal frame.

### Claim 11 (depends on 8)

The apparatus according to claim 8, wherein the processor is further configured to:
divide the first speech signal into the multiple speech signal frames according to frame duration determined based on a Haas effect and a preset sampling frequency;
add an analysis window ha to each speech signal frame in the multiple speech signal frames, to obtain multiple first windowed signals; and perform discrete Fourier transform on each first windowed signal frame in the multiple first windowed signals to obtain the multiple first frequency domain signal frames;
perform inverse discrete Fourier transform on each second frequency domain signal frame in the multiple second frequency domain signal frames, to obtain multiple first time domain signal frames; add a synthesis window hs to each first time domain signal frame in the multiple first time domain signal frames, to obtain multiple second windowed signal frames; and perform frame fusion on the multiple second windowed signal frames, to obtain the second speech signal.

### Claim 12 (depends on 11)

The apparatus according to claim 11, wherein a frame length N of each speech signal frame in the multiple speech signal frames, the analysis window ha and the synthesis window hs, satisfy the following relationship:
∑
n
=
0
N
-
1
h
a
(
n
)
⁢
h
s
(
n
)
=
1
wherein ha(n) represents an nth item in the analysis window ha, and hs(n) represents an nth item in the synthesis window hs.

### Claim 13 (depends on 8)

The apparatus according to claim 8, wherein the target frequency threshold being related to the sampling frequency of the second device comprises:
the target frequency threshold being the same as an upper limit of the sampling frequency of the second device.

### Claim 14

A non-transitory computer-readable storage medium, storing computer-readable instructions, the computer-readable instructions, when executed by a processor, causing the processor to perform operations comprising:
obtaining a first speech signal acquired by a first device;
performing frame blocking on the first speech signal, to obtain multiple speech signal frames;
converting the multiple speech signal frames into multiple first frequency domain signal frames;
performing aliasing processing on a first sub-frequency domain signal frame with a frequency lower than or equal to a target frequency threshold in each first frequency domain signal frame in the multiple first frequency domain signal frames, and reserving a second sub-frequency domain signal frame among the multiple first frequency domain signal frames with a frequency higher than the target frequency threshold in the each first frequency domain signal frame, to obtain multiple second frequency domain signal frames, the target frequency threshold being related to a sampling frequency of a second device, comprising:
alternately enhancing and attenuating the first sub-frequency domain signal in the each first frequency domain signal frame, to obtain a third sub-frequency domain signal frame, and reserving the second sub-frequency domain signal in the each first frequency domain signal frame, to obtain the multiple second frequency domain signal frames, wherein a kth second frequency domain signal frame Xw′[k,n] of the multiple second frequency domain signal frames equals to the third sub-frequency domain signal frame obtained after aliasing processing is performed on the first sub-frequency domain signal frame in a kth first frequency domain signal frame of the first frequency domain signal frames when a frequency value n is less than or equal to the target frequency threshold; and
performing frame fusion on the multiple second frequency domain signal frames, to obtain a second speech signal.

### Claim 15 (depends on 2)

The method according to claim 2, wherein the low-frequency aliasing function H[k,n] of the kth frame satisfies the following relationship:
H
[
2
⁢
m
+
1
,
n
]
=
1
H
[
2
⁢
m
,
n
]
H
[
2
⁢
m
,
n
]
=
𝒟
[
m
,
n
]
wherein [m,n] is a filter, H[k,n] when k is an odd number is a reciprocal of H[k,n] when k is an even number, and the k includes 2m+1 and 2m.

### Claim 16 (depends on 2)

The method according to claim 2, wherein the low-frequency aliasing function H[k,n] of the kth frame satisfies the following relationship:
H
[
2
⁢
m
+
1
,
n
]
=
1
H
[
2
⁢
m
,
n
]
H
[
2
⁢
m
,
n
]
=
𝒟
[
m
,
n
]
wherein [m, n] is a filter, H[k,n] when k is an odd number is a negative number of H[k,n] when k is an even number, and the k includes 2m+1 and 2m.

### Claim 17 (depends on 4)

The method according to claim 4, wherein a kth first windowed signal frame in the multiple first windowed signals satisfies the following relationship:
h
a
[
n
]
=
{
1
2
⁢
(
1
-
cos
⁢
2
⁢
π
⁢
n
N
-
1
)
,
0
≤
n
≤
N
-
1
0
,
n
<
0
⁢
or
⁢
n
>
N
-
1
wherein ha [n] represents an nth item in the analysis window ha, and ha [n−kM] represents an (n−kM)th item in the analysis window ha.

### Claim 18 (depends on 4)

The method according to claim 4, wherein a second windowed signal frame in the multiple second windowed signal frames satisfies the following relationship:
x
′
[
n
]
=
∑
k
x
w
′
[
k
,
n
-
k
⁢
M
]
⁢
h
s
[
n
-
k
⁢
M
]
wherein M represents a quantity of overlapping sampling points of adjacent frames in the multiple speech signal frames, xw′[k, n−kM] represents an (n−kM)th value in a kth first time domain signal frame, and x′[n] represents an nth value of the second speech signal; and
the synthesis window hs comprises:
h
s
[
n
]
=
{
1
,
0
≤
n
≤
N
-
1
0
,
n
<
0
⁢
or
⁢
n
>
N
-
1
wherein hs[n] represents an nth item in the synthesis window hs, and hs[n−kM] represents an (n−kM)th item in the synthesis window hs.