Back to results
US 12,763,063 B2Granted
Information processing device and information processing method for artificial speech generation
AssigneeSony Group Corporation
InventorJustinas Miseikis
Life of this patent
Estimating the term…
Abstract
An information processing device for generating artificial speech data has circuitry which is configured to obtain, based on speech data, speech emotional indicators and associated timing data of the emotional indicators; and to obtain, based on the speech data, text data; and to generate artificial speech data based on the text data, the speech emotional indicators and the associated timing data.
Claims18
An information processing device for generating artificial speech data, comprising circuitry configured to:
- obtain, based on speech data, speech emotional indicators and associated timing data of the emotional indicators;
- obtain, based on the speech data, text data;
- obtain, based on video data associated with the speech data, video emotional indicators comprising at least one of facial expressions, eye direction, gestures, and body pose;
- associate the speech emotional indicators and the video emotional indicators with the text data based on the associated timing data;
- encode the speech emotional indicators and the video emotional indicators into a synchronized latent feature space matched to timestamps of the text data:
- generate artificial speech data based on the synchronized latent feature space, the text data, the speech emotional indicators associated with the text data, the video emotional indicators associated with the text data, and the associated timing data.
3 more elements
An information processing method for generating artificial speech data, comprising:
- obtaining, based on speech data, speech emotional indicators and associated timing data of the emotional indicators;
- obtaining, based on the speech data, text data;
- obtaining, based on video data associated with the speech data, video emotional indicators comprising at least one of facial expressions, eye direction, gestures, and body pose;
- associating the speech emotional indicators and the video emotional indicators with the text data based on the associated timing data;
- encoding the speech emotional indicators and the video emotional indicators into a synchronized latent feature space matched to timestamps of the text data;
- and
- generating artificial speech data based on the synchronized latent feature space, the text data, the speech emotional indicators associated with the text data, the video emotional indicators associated with the text data, and the associated timing data.
4 more elements
Dependencies are inferred from the claim text.