Back to results
US 10,832,660 B2Granted
Method and device for processing whispered speech
AssigneeFuturewei Technologies, Inc.
InventorAmin Fazeldehkordi
Life of this patent
Estimating the term…
Abstract
The present disclosure provides embodiments of method, system and devices to convert whispered speech to normal speech intelligently through deep learning, therefore whispered speech can be more robust to interference and more intelligible to listener. A first audio signal is received including first whispered speech; a first plurality of computations are performed on the first audio signal to extract a first features; the first features are provided as input to a trained deep neural network (DNN) model to obtain output of the DNN model comprising a second features; and an inverse of the first plurality of computations are performed on the second features to produce a second audio signal corresponding to a non-whispered version of the first whispered speech.
Claims20
A method of processing whispered speech comprising:
- receiving a first audio signal comprising first whispered speech;
- performing a first plurality of computations on the first audio signal to extract a plurality of first features;
- providing the first features as input to a trained deep neural network (DNN) model to obtain output of the DNN model comprising a plurality of second features, wherein the DNN was trained by:
- obtaining a plurality of different training pairs, wherein each training pair comprises first features of a respective audio signal comprising whispered speech and second features of a respective audio signal comprising a non-whispered version of the whispered speech;
- and
- for each of the training pairs:
- supplying the first features of the pair as input to the DNN model and comparing the output of the DNN to the second features of the pair using a loss function that produces an output;
- and
- applying back-propagation to the DNN model based on the output of the loss function to train the DNN model, wherein the loss function is:
- ∑
- n
- =
- 1
- N
-
-
- Y
- ^
- n
-
- (
- W
- ,
- b
- )
- -
- Y
- n
-
- 2
- 2
- ,
- wherein
- N indicates total number of a plurality of neurons in the DNN model,
- n indicates count of calculation of the loss function,
- W indicates a combination of a plurality of weights,
- b indicates a plurality of biases which are constant number,
- Y indicates the second features of the training pair,
- Ŷ indicates the first features of the training pair;
- and
- performing an inverse of the first plurality of computations on the plurality of second features to produce a second audio signal corresponding to a non-whispered version of the first whispered speech.
38 more elements
A device of processing whispered speech comprising:
- a memory storing instructions;
- and
- a processor coupled to the memory, wherein when the processor executes the instructions, the processor performs operations comprising:
- receiving a first audio signal comprising first whispered speech;
- performing a first plurality of computations on the first audio signal to extract a plurality of first features;
- providing the first features as input to a trained deep neural network (DNN) model to obtain output of the DNN model comprising a plurality of second features wherein the DNN was trained by:
- obtaining a plurality of different training pairs, wherein each training pair comprises first features of a respective audio signal comprising whispered speech and second features of a respective audio signal comprising a non-whispered version of the whispered speech;
- and
- for each of the training pairs:
- supplying the first features of the pair as input to the DNN model and comparing the output of the DNN to the second features of the pair using a loss function that produces an output;
- and
- applying back-propagation to the DNN model based on the output of the loss function to train the DNN model, wherein the loss function is:
- ∑
- n
- =
- 1
- N
-
-
- Y
- ^
- n
-
- (
- W
- ,
- b
- )
- -
- Y
- n
-
- 2
- 2
- ,
- wherein
- N indicates total number of a plurality of neurons in the DNN model,
- n indicates count of calculation of the loss function,
- W indicates a combination of a plurality of weights,
- b indicates a plurality of biases which are constant number,
- Y indicates the second features of the training pair,
- Ŷ indicates the first features of the training pair;
- and
- performing an inverse of the first plurality of computations on the second features to produce a second audio signal corresponding to a non-whispered version of the first whispered speech.
41 more elements
A non-transitory, computer-readable medium storing one or more instructions executable by a computer system to perform operations comprising:
- receiving a first audio signal comprising first whispered speech;
- performing a first plurality of computations on the first audio signal to extract a plurality of first features;
- providing the first features as input to a trained deep neural network (DNN) model to obtain output of the DNN model comprising a plurality of second features, wherein the DNN was trained by:
- obtaining a plurality of different training pairs, wherein each training pair comprises first features of a respective audio signal comprising whispered speech and second features of a respective audio signal comprising a non-whispered version of the whispered speech;
- and
- for each of the training pairs:
- supplying the first features of the pair as input to the DNN model and comparing the output of the DNN to the second features of the pair using a loss function that produces an output;
- and
- applying back-propagation to the DNN model based on the output of the loss function to train the DNN model, wherein the loss function is:
- ∑
- n
- =
- 1
- N
-
-
- Y
- ^
- n
-
- (
- W
- ,
- b
- )
- -
- Y
- n
-
- 2
- 2
- ,
- wherein
- N indicates total number of a plurality of neurons in the DNN model,
- n indicates count of calculation of the loss function,
- W indicates a combination of a plurality of weights,
- b indicates a plurality of biases which are constant number,
- Y indicates the second features of the training pair,
- Ŷ indicates the first features of the training pair;
- and
- performing an inverse of the first plurality of computations on the second features to produce a second audio signal corresponding to a non-whispered version of the first whispered speech.
38 more elements
Dependencies are inferred from the claim text.