# US 12,223,972 B2 · Voice processing method and apparatus, electronic device, and computer-readable storage medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,223,972 B2
- Document ID: US12223972B2
- Status: Granted
- Kind: B2
- Application number: 17/714,485
- Filed: 2022-04-06
- Published: 2022-07-21
- Granted: 2025-02-11
- Earliest claimed: 2020-05-15
- Assignees: Tencent Technology (Shenzhen) Company Limited
- Applicants: Tencent Technology (Shenzhen) Company Limited
- Inventors: Wei Xiao; Meng Wang; Ling Zhu; Wenlong Wang
- Web page: https://patentgrep.com/patent/US12223972B2

## CPC

- G10L 19/005: Correction of errors induced by the transmission channel, if related to the coding algorithm
- G10L 21/00: Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility (G10L19/00 takes precedence)
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/045: Combinations of networks
- G06N 3/0499: Feedforward networks
- G06N 3/09: Supervised learning
- G10L 19/04: using predictive techniques
- G10L 25/18: the extracted parameters being spectral information of each sub-band
- G10L 25/30: using neural networks
- H04L 65/60: Network streaming of media packets
- H04L 65/70: Media network packetisation
- H04L 65/765: {intermediate}
- H04L 65/80: Responding to QoS
- G06N 3/08: Learning methods
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

A voice processing method includes: determining a historical voice frame corresponding to a target voice frame; acquiring a frequency-domain characteristic of the historical voice frame and a time-domain parameter of the historical voice frame; obtaining a parameter set of the target voice frame according to a correlation between the frequency-domain characteristic of the historical voice frame and the time-domain parameter of the historical voice frame, the parameter set including at least two parameters; and reconstructing the target voice frame according to the parameter set.

## Claims

### Claim 1

A voice processing method, comprising:
determining a historical voice frame corresponding to a target voice frame;
acquiring a frequency-domain characteristic of the historical voice frame and a time-domain parameter of the historical voice frame;
obtaining a parameter set of the target voice frame according to a correlation between the frequency-domain characteristic of the historical voice frame and the time-domain parameter of the historical voice frame, the parameter set including at least two parameters; and
reconstructing the target voice frame according to the parameter set,
wherein the network model includes a first neural network and at least two second neural networks, the first neural network is in a cascade relationship with each of the second neural networks, and one of the second neural networks corresponding to a parameter in the parameter set, and obtaining the parameter set of the target voice frame comprises:
invoking the first neural network based on the frequency-domain characteristic of the historical voice frame, to obtain a virtual frequency-domain characteristic of the target voice frame; and
inputting the virtual frequency-domain characteristic of the target voice frame and the time-domain parameter of the historical voice frame to the at least two second neural networks respectively as input information, to obtain the at least two parameters in the parameter set of the target voice frame.

### Claim 2 (depends on 1)

The method according to claim 1, wherein reconstructing the target voice frame comprises:
establishing a reconstruction filter according to the parameter set;
acquiring an excitation signal of the target voice frame; and
filtering the excitation signal of the target voice frame by using the reconstruction filter to obtain the target voice frame.

### Claim 3 (depends on 2)

The method according to claim 2, wherein acquiring the excitation signal of the target voice frame comprises:
acquiring an excitation signal of the historical voice frame; and
determining the excitation signal of the target voice frame according to the excitation signal of the historical voice frame.

### Claim 4 (depends on 3)

The method according to claim 3, wherein the target voice frame is an nth voice frame in a voice signal transmitted by a Voice over Internet Protocol (VOIP) system, and the historical voice frame includes an (n−t)th voice frame to an (n−1)th voice frame in the voice signal transmitted by the VoIP system, n and t being both positive integers.

### Claim 5 (depends on 4)

The method according to claim 4, wherein
the excitation signal of the historical voice frame includes an excitation signal of the (n−1)th voice frame; and determining the excitation signal of the target voice frame comprises:
determining the excitation signal of the (n−1)th voice frame as the excitation signal of the target voice frame.

### Claim 6 (depends on 4)

The method according to claim 4, wherein the excitation signal of the historical voice frame includes excitation signals of voice frames from the (n−t)th voice frame to the (n−1)th voice frame; and determining the excitation signal of the target voice frame comprises:
averaging the excitation signals of the (n−t)th voice frame to the (n−1)th voice frame to obtain the excitation signal of the target voice frame.

### Claim 7 (depends on 4)

The method according to claim 4, wherein the excitation signal of the historical voice frame includes excitation signals of voice frames from the (n−t)th voice frame to the (n−1)th voice frame; and determining the excitation signal of the target voice frame comprises:
performing weighted summation on the excitation signals of the (n−t)th voice frame to the (n−1)th voice frame to obtain the excitation signal of the target voice frame.

### Claim 8 (depends on 1)

The method according to claim 1, wherein acquiring the frequency-domain characteristic comprises:
performing short-term Fourier transform (STFT) on the historical voice frame to obtain a frequency-domain coefficient corresponding to the historical voice frame; and
extracting an amplitude spectrum from the frequency-domain coefficient corresponding to the historical voice frame as the frequency-domain characteristic of the historical voice frame.

### Claim 9 (depends on 2)

The method according to claim 2, wherein in response to determining that the target voice frame is an unvoiced frame, the time-domain parameter of the historical voice frame includes a short-term correlation parameter of the historical voice frame, and the parameter set includes a short-term correlation parameter of the target voice frame; and the reconstruction filter includes a linear predictive coding (LPC) filter; and
the target voice frame includes k daughter frames, and the short-term correlation parameter of the target voice frame includes a line spectral frequency (LSF) and an interpolation factor of a kth daughter frame of the target voice frame, k being an integer greater than 1.

### Claim 10 (depends on 2)

The method according to claim 2, wherein in response to determining that the target voice frame is a voiced frame, the time-domain parameter of the historical voice frame includes a short-term correlation parameter and a long-term correlation parameter of the historical voice frame, and the parameter set includes a short-term correlation parameter of the target voice frame and a long-term correlation parameter of the target voice frame; the reconstruction filter includes a long-term prediction (LTP) filter and an LPC filter;
the target voice frame includes k daughter frames, and the short-term correlation parameter of the target voice frame includes an LSF and an interpolation factor of a kth daughter frame of the target voice frame, k being an integer greater than 1; and
the target voice frame includes m subframes, and the long-term correlation parameter of the target voice frame includes a pitch lag and an LTP coefficient of each subframe of the target voice frame, m being a positive integer.

### Claim 11 (depends on 1)

The method according to claim 1, wherein the network model further includes a third neural network, a network formed by the first neural network, each of the second neural networks, and the third neural network being a parallel network; and the time-domain parameter of the historical voice frame includes an energy parameter of the historical voice frame; and
obtaining the parameter set of the target voice frame comprises:
inputting the virtual frequency-domain characteristic of the target voice frame and the energy parameter of the historical voice frame to the at least two second neural networks respectively as input information, to obtain the at least two parameters of the target voice frame;
invoking the third neural network based on the energy parameter of the historical voice frame, to obtain an energy parameter of the target voice frame; and
forming the parameter set of the target voice frame by the at least two parameters of the target voice frame and the energy parameter of the target voice frame,
the target voice frame including m subframes, and the energy parameter of the target voice frame including a gain value of each of the subframes of the target voice frame.

### Claim 12

A voice processing apparatus, comprising: a memory storing computer program instructions; and a processor coupled to the memory and configured to execute the computer program instructions and perform:
determining a historical voice frame corresponding to a target voice frame;
acquiring a frequency-domain characteristic of the historical voice frame and a time-domain parameter of the historical voice frame;
obtaining a parameter set of the target voice frame according to a correlation between the frequency-domain characteristic of the historical voice frame and the time-domain parameter of the historical voice frame, the parameter set including at least two parameters; and
reconstructing the target voice frame according to the parameter set,
wherein the network model includes a first neural network and at least two second neural networks, the first neural network is in a cascade relationship with each of the second neural networks, and one of the second neural networks corresponding to a parameter in the parameter set, and obtaining the parameter set of the target voice frame comprises:
invoking the first neural network based on the frequency-domain characteristic of the historical voice frame, to obtain a virtual frequency-domain characteristic of the target voice frame; and
inputting the virtual frequency-domain characteristic of the target voice frame and the time-domain parameter of the historical voice frame to the at least two second neural networks respectively as input information, to obtain the at least two parameters in the parameter set of the target voice frame.

### Claim 13 (depends on 12)

The voice processing apparatus according to claim 12, wherein reconstructing the target voice frame includes:
establishing a reconstruction filter according to the parameter set;
acquiring an excitation signal of the target voice frame; and
filtering the excitation signal of the target voice frame by using the reconstruction filter to obtain the target voice frame.

### Claim 14 (depends on 13)

The voice processing apparatus according to claim 13, wherein acquiring the excitation signal of the target voice frame includes:
acquiring an excitation signal of the historical voice frame; and
determining the excitation signal of the target voice frame according to the excitation signal of the historical voice frame.

### Claim 15 (depends on 14)

The voice processing apparatus according to claim 14, wherein the target voice frame is an nth voice frame in a voice signal transmitted by a Voice over Internet Protocol (VOIP) system, and the historical voice frame includes an (n−t)th voice frame to an (n−1)th voice frame in the voice signal transmitted by the VoIP system, n and t being both positive integers.

### Claim 16 (depends on 15)

The voice processing apparatus according to claim 15, wherein
the excitation signal of the historical voice frame includes an excitation signal of the (n−1)th voice frame; and determining the excitation signal of the target voice frame comprises:
determining the excitation signal of the (n−1)th voice frame as the excitation signal of the target voice frame.

### Claim 17 (depends on 15)

The voice processing apparatus according to claim 15, wherein the excitation signal of the historical voice frame includes excitation signals of voice frames from the (n−t)th voice frame to the (n−1)th voice frame; and determining the excitation signal of the target voice frame comprises:
averaging the excitation signals of the (n−t)th voice frame to the (n−1)th voice frame to obtain the excitation signal of the target voice frame.

### Claim 18

A non-transitory computer-readable storage medium storing computer program instructions executable by at least one processor to perform:
determining a historical voice frame corresponding to a target voice frame;
acquiring a frequency-domain characteristic of the historical voice frame and a time-domain parameter of the historical voice frame;
obtaining a parameter set of the target voice frame according to a correlation between the frequency-domain characteristic of the historical voice frame and the time-domain parameter of the historical voice frame, the parameter set including at least two parameters; and
reconstructing the target voice frame according to the parameter set,
wherein the network model includes a first neural network and at least two second neural networks, the first neural network is in a cascade relationship with each of the second neural networks, and one of the second neural networks corresponding to a parameter in the parameter set, and obtaining the parameter set of the target voice frame comprises:
invoking the first neural network based on the frequency-domain characteristic of the historical voice frame, to obtain a virtual frequency-domain characteristic of the target voice frame; and
inputting the virtual frequency-domain characteristic of the target voice frame and the time-domain parameter of the historical voice frame to the at least two second neural networks respectively as input information, to obtain the at least two parameters in the parameter set of the target voice frame.