Audio translator
AssigneeSpotify AB
InventorMarco Marchini
Life of this patent
Estimating the term…
Abstract
Audio translation system includes a feature extractor and a style transfer machine learning model. The feature extractor generates for each of a plurality of source voice files one or more source voice parameters encoded as a collection of source feature vectors, and generates for each of a plurality of target voice files one or more target voice parameters encoded as a collection of target feature vectors. The style transfer machine learning model trained on the collection of source feature vectors for the plurality of source voice files and the collection of target feature vectors for the plurality of target voice files to generate a style transformed feature vector.
Claims20
A method for performing audio translation, the method comprising:
- receiving a sample voice file;
- displaying one or more feature curves of the sample voice file on a user interface;
- displaying a plurality of styles;
- receiving a selected style from the plurality of styles;
- generating, from the sample voice file, a sample tensor identifying the presence of a salient auditory feature over time;
- cropping the sample tensor along a time dimension and a frequency dimension using a sliding window, thereby obtaining at least one cropped sample tensor;
- applying a machine learning model on each cropped sample tensor of the at least one cropped sample tensor, thereby generating a collection of resulting cropped tensors;
- stitching the collection of resulting cropped tensors, thereby forming a transformed sample tensor;
- translating content of the sample voice file from a first domain style to a second domain style based on an application of one or more feature vectors of the selected style to the one or more feature curves, wherein translating the content of the sample voice file from the first domain style to the second domain style is based on a difference between the sample tensor and the transformed sample tensor;
- and
- displaying a representation of translated content of the sample voice file in the second domain style on the user interface.
A non-transitory computer-readable medium having stored thereon one or more sequences of instructions for causing one or more processors to perform:
- receiving a sample voice file;
- displaying one or more feature curves of the sample voice file on a user interface;
- displaying a plurality of styles;
- receiving a selected style from the plurality of styles;
- generating, from the sample voice file, a sample tensor identifying the presence of a salient auditory feature over time;
- cropping the sample tensor along a time dimension and a frequency dimension using a sliding window, thereby obtaining at least one cropped sample tensor;
- applying a machine learning model on each cropped sample tensor of the at least one cropped sample tensor, thereby generating a collection of resulting cropped tensors;
- stitching the collection of resulting cropped tensors, thereby forming a transformed sample tensor;
- translating content of the sample voice file from a first domain style to a second domain style based on an application of one or more feature vectors of the selected style to the one or more feature curves, wherein translating the content of the sample voice file from the first domain style to the second domain style is based on a difference between the sample tensor and the transformed sample tensor;
- and
- displaying a representation of translated content of the sample voice file in the second domain style on the user interface.
A system for audio translation, the system comprising:
- a user interface;
- and
- a sample audio processor configured to:
- receive a sample voice file;
- display one or more feature curves of the sample voice file on the user interface;
- display a plurality of styles;
- receive a selected style from the plurality of styles;
- generate, from the sample voice file, a sample tensor identifying the presence of a salient auditory feature over time;
- crop the sample tensor along a time dimension and a frequency dimension using a sliding window, thereby obtaining at least one cropped sample tensor;
- apply a machine learning model on each cropped sample tensor of the at least one cropped sample tensor, thereby generating a collection of resulting cropped tensors;
- stitch the collection of resulting cropped tensors, thereby forming a transformed sample tensor;
- translate content of the sample voice file from a first domain style to a second domain style based on an application of one or more feature vectors of the selected style to the one or more feature curves, wherein the sample audio processor translates the content of the sample voice file from the first domain style to the second domain style based on a difference between the sample tensor and the transformed sample tensor;
- and
- display a representation of translated content of the sample voice file in the second domain style on the user interface.
Dependencies are inferred from the claim text.