Back to results
US 11,900,914 B2Granted
User self-personalized text-to-speech voice generation
AssigneeMeta Platforms, Inc.
InventorsMaria Fernandez Guajardo, Trent Richard Walkiewicz
Life of this patent
Estimating the term…
Abstract
An online system receives, from a client device of a posting user, a script for a voice-based content item. The online system retrieves a voice synthesis model stored in the user profile of the posting user and generates a synthetic audio stream using the retrieved voice synthesis model and based on the received script. The online system presents the generated synthetic audio stream to the posting user and receives instructions for modifying the synthetic audio stream. The online system generates a second audio stream based on the received instructions and composes the voice-based tent item based on the generated second audio stream.
Claims17
A method comprising:
- receiving, from a client device of a posting user of an online system, a script for a voice-based content item;
- retrieving a voice synthesis model stored in a user profile of the posting user, the voice synthesis model trained at least based on a plurality of voice samples of the posting user, wherein the script comprises an indication of a mood for the voice-based content item, and wherein retrieving the voice synthesis model comprises selecting the voice synthesis model from a plurality of candidate voice synthesis models stored in the user profile of the posting user, each different candidate voice synthesis models trained using training data corresponding to a different mood, the selection of the voice synthesis model selected from the plurality of candidate voice synthesis models based on the indication of the mood for the voice-based content item matching a mood with which the voice synthesis model's training data was labeled;
- generating a synthetic audio stream using the retrieved voice synthesis model and based on the received script;
- presenting the generated synthetic audio stream to the posting user;
- receiving instructions for modifying the synthetic audio stream;
- generating a second audio stream based on the received instructions;
- composing the voice-based content item based on the generated second audio stream;
- and
- presenting the voice-based content item to a viewing user of the online system.
6 more elements
A non-transitory computer-readable storage medium configured to store instructions, the instructions when executed by a processor cause the processor to:
- receive, from a client device of a posting user of an online system, a script for a voice-based content item;
- retrieve a voice synthesis model stored in a user profile of the posting user, the voice synthesis model trained at least based on a plurality of voice samples of the posting user, wherein the script comprises an indication of a mood for the voice-based content item, and wherein retrieving the voice synthesis model comprises selecting the voice synthesis model from a plurality of candidate voice synthesis models stored in the user profile of the posting user, each different candidate voice synthesis models trained using training data corresponding to a different mood, the selection of the voice synthesis model selected from the plurality of candidate voice synthesis models based on the indication of the mood for the voice-based content item matching a mood with which the voice synthesis model's training data was labeled;
- generate a synthetic audio stream using the retrieved voice synthesis model and based on the received script;
- present the generated synthetic audio stream to the posting user;
- receive instructions for modifying the synthetic audio stream;
- generate a second audio stream based on the received instructions;
- compose the voice-based content item based on the generated second audio stream;
- and
- present the voice-based content item to a viewing user of the online system.
6 more elements
A system comprising:
- a user profile store configured to store a user profile for each user of an online system, each user profile comprising a set of voice synthesis model trained at least based on a plurality of voice samples of the user;
- and
- a voice-based content item generator configured to:
- receive, from a client device of a posting user of an online system, a script for a voice-based content item;
- retrieve a voice synthesis model stored in the user profile of the posting user, wherein the script comprises an indication of a mood for the voice-based content item, and wherein retrieving the voice synthesis model comprises selecting the voice synthesis model from a plurality of candidate voice synthesis models stored in the user profile of the posting user, each different candidate voice synthesis models trained using training data corresponding to a different mood, the selection of the voice synthesis model selected from plurality set of candidate voice synthesis models based on the indication of the mood for the voice-based content item matching a mood with which the voice synthesis model's training data was labeled;
- generate a synthetic audio stream using the retrieved voice synthesis model and based on the received script;
- present the generated synthetic audio stream to the posting user;
- receive instructions for modifying the synthetic audio stream;
- generate a second audio stream based on the received instructions;
- and
- compose the voice-based content item based on the generated second audio stream.
8 more elements
Dependencies are inferred from the claim text.