Method and system for creating a prosodic script

An apparatus for creating a script for rendering audio and/or video streams, comprising:

• a processor;
• and
• a memory accessible to the processor, the memory having stored therein at least one of programs or instructions executable by the processor to configure the apparatus to:
• identify at least one prosodic speech feature, in at least one of a received audio stream or a received language model, and/or at least one prosodic gesture in a received video stream;
• and
• automatically temporally annotate an associated text stream with at least one prosodic speech symbol created from the identified at least one prosodic speech feature and/or at least one prosodic gesture symbol created from the identified at least prosodic gesture to create a prosodic script, that when rendered, provides an audio stream and/or a video stream comprising the at least one prosodic speech feature and/or the at least one prosodic gesture that are temporally aligned.

A system for creating a script for rendering audio and/or video streams, comprising:

• a spectral features module;
• a gesture features module;
• a streams to script module;
• and
• an apparatus comprising a processor and a memory accessible to the processor, the memory having stored therein at least one of programs or instructions executable by the processor to configure the apparatus to:
• identify, using the spectral features module and/or the gesture features module, at least one prosodic speech feature, in at least one of a received audio stream or a received language model, and/or at least one prosodic gesture in a received video stream;
• and
• automatically temporally annotate, using the streams to script module, an associated text stream with at least one prosodic speech symbol created from the identified at least one prosodic speech feature and/or at least one prosodic gesture symbol created from the identified at least prosodic gesture to create a prosodic script, that when rendered, provides an audio stream and/or a video stream comprising the at least one prosodic speech feature and/or the at least one prosodic gesture that are temporally aligned.

A method for creating a dynamic prosodic script for rendering audio and/or video streams, comprising:

• identifying at least one prosodic speech feature in a received audio stream and/or a received language model;
• creating at least one modifiable prosodic speech symbol for each of the identified at least one prosodic speech features;
• converting the received audio stream and/or the language model into a text stream;
• automatically, temporally annotating the text stream with at least one created, modifiable prosodic speech symbol;
• identifying in a received video stream at least one prosodic gesture of at least a portion of a body of a speaker of the received audio stream;
• creating at least one modifiable prosodic gesture symbol for each of the identified at least one prosodic gestures;
• and
• temporally and automatically annotating the text stream with at least one created, modifiable prosodic gesture symbol along with the at least one modifiable, prosodic speech symbol to create a prosodic script, wherein the at least one modifiable prosodic speech symbol and the at least one modifiable prosodic gesture symbol are modifiable in the prosodic script, such that a rendering of an audio stream or a video stream from the prosodic script is changed when at least one of the at least one modifiable prosodic speech symbol and the at least one modifiable prosodic gesture symbol is modified.