Training and testing utterance-based frameworks
AssigneeSpotify AB
InventorDaniel Bromand
Life of this patent
Estimating the term…
Abstract
Systems, methods, and devices for training and testing utterance based frameworks are disclosed. The training and testing can be conducting using synthetic utterance samples in addition to natural utterance samples. The synthetic utterance samples can be generated based on a vector space representation of natural utterances. In one method, a synthetic weight vector associated with a vector space is generated. An average representation of the vector space is added to the synthetic weight vector to form a synthetic feature vector. The synthetic feature vector is used to generate a synthetic voice sample. The synthetic voice sample is provided to the utterance-based framework as at least one of a testing or training sample.
Claims18
A method for training an audio voice framework, the method comprising:
- generating a set of natural voice samples as voice training samples from one or more audio clips of the speech of one or more individuals;
- generating a set of synthetic voice samples by playing one or more audio voice samples through a speaker to an appliance;
- inputting the set of natural voice samples and the set of synthetic voice samples to an audio voice framework to produce test voice samples;
- comparing the produced test voice samples against an expected test voice samples result;
- if the produced test voice samples do not match the expected test samples result:
- determining from the produced test voice samples a plurality of the input natural voice samples and synthetic voice samples that caused the audio voice framework to produce incorrect test voice samples output;
- generating new synthetic voice samples by modifying one or more of the plurality of input natural voice samples and synthetic voice samples that caused the incorrect output;
- and
- adding the generated new synthetic voice samples to the set of synthetic voice samples to be input to the audio voice framework to retrain the audio voice framework.
A system for training an audio voice framework, comprising:
- one or more processors;
- and
- a computer-readable storage medium coupled to the one or more processors and comprising instructions thereon that, when executed by the one or more processors, cause the one or more processors to:
- generate a set of natural voice samples as voice training samples from one or more audio clips of the speech of one or more individuals;
- generate a set of synthetic voice samples by playing one or more audio voice samples through a speaker to an appliance;
- input the set of natural voice samples and the set of synthetic voice samples to an audio voice framework to produce test voice samples;
- compare the produced test samples against an expected test voice samples result;
- if the produced test voice samples do not match the expected test voice samples result:
- determine from the produced test voice samples a plurality of the input natural voice samples and synthetic voice samples that caused the audio voice framework to produce incorrect test voice samples output;
- generate new synthetic voice samples by modifying one or more of the plurality of input natural voice samples and synthetic voice samples that caused the incorrect output;
- and
- add the generated new synthetic voice samples to the set of synthetic voice samples to be input to the audio voice framework to retrain the audio voice framework.
A non-transitory computer readable medium having instructions stored thereon that, when executed by one or more processors, cause the one or more processors to:
- generate a set of natural voice samples as voice training samples from one or more audio clips of the speech of one or more individuals;
- generate a set of synthetic voice samples by playing one or more audio voice samples through a speaker to an appliance;
- input the set of natural voice samples and the set of synthetic voice samples to an audio voice framework to produce test voice samples;
- compare the produced test samples against an expected test voice samples result;
- if the produced test voice samples do not match the expected test voice samples result:
- determine from the produced test voice samples a plurality of the input natural voice samples and synthetic voice samples that caused the audio voice framework to produce incorrect test voice samples output;
- generate new synthetic voice samples by modifying one or more of the plurality of input natural voice samples and synthetic voice samples that caused the incorrect output;
- and
- add the generated new synthetic voice samples to the set of synthetic voice samples to be input to the audio voice framework to retrain the audio voice framework.
Dependencies are inferred from the claim text.