US 11,321,890 B2Granted

User interface for generating expressive content

AssigneeMicrosoft Technology Licensing, LLC

InventorsAnn M. Paradiso, Jonathan Campbell, Edward Bryan Cutrell, Harish Kulkarni, Meredith Morris, Alexander John Fiannaca, Kiley Rebecca Sobel

Life of this patent

Estimating the term…

patentgrep

Loading patent…

Abstract

Generation of expressive content is provided. An expressive synthesized speech system provides improved voice authoring user interfaces by which a user is enabled to efficiently author content for generating expressive output. An expressive synthesized speech system provides an expressive keyboard for enabling input of textual content and for selecting expressive operators, such as emoji objects or punctuation objects for applying predetermined prosody attributes or visual effects to the textual content. A voicesetting editor mode enables the user to author and adjust particular prosody attributes associated with the content for composing carefully-crafted synthetic speech. An active listening mode (ALM) is provided, which when selected, a set of ALM effect options are displayed, wherein each option is associated with a particular sound effect and/or visual effect. The user is enabled to rapidly respond with expressive vocal sound effects or visual effects while listening to others speak.

Claims19

Dependent claims

A computer-implemented method for generating expressive content comprising:

displaying an expressive keyboard, wherein the expressive keyboard includes an alpha-numeric keyboard for receiving textual input and a plurality of emoji objects for selectively applying a vocal sound effect associated with each of the plurality of emoji objects to received textual input;
receiving textual input;
displaying on the expressive keyboard a set of the plurality of emoji objects, based on linguistic properties of the received textual input;
selecting, by a user, at least one of the displayed emoji objects;
in response to receiving the selection of the at least one emoji object, identifying a predefined vocal sound effect associated with the selected emoji object;
combining the identified vocal sound effect with the received textual input;
outputting the combined identified vocal sound effect and received textual input to a speech generation engine for generating expressive synthesized speech;
providing a voicesetting editor interface;
in response to receiving a selection to launch the voicesetting editor interface:
parsing the received textual input and any selected emoji objects;
and
displaying the parsed textual input and any selected emoji objects as selectable tokens;
in response to receiving a selection of a token, displaying a set of prosodic properties that can be applied to the selected token;
and
in response to receiving a selection of a prosodic property, displaying a value associated with the selected prosodic property for allowing a user to adjust the value for controlling expressivity of the textual input when rendered.

12 more elements

A system for generating expressive content, the computing device comprising:

at least one processing device;
and
at least one computer readable data storage device storing instructions that, when executed by the at least one processing device, cause the computing device to provide an expressive synthesized speech system, the expressive synthesized speech system operative to:
display an expressive keyboard, wherein the expressive keyboard includes an alpha-numeric keyboard for receiving textual input and a plurality of emoji objects for selectively applying a vocal sound effect associated with each of the plurality of emoji objects to received textual input;
receive textual input;
display on the expressive keyboard a set of the plurality of emoji objects, based on linguistic properties of the received textual input;
select, by a user, at least one of the displayed emoji objects;
in response to receiving the selection of the at least one emoji object, identify a predefined vocal sound effect associated with the selected emoji object;
combine the identified vocal sound effect with the received textual input;
output the combined set of identified vocal sound effect and received textual input to a speech generation engine for generating expressive synthesized speech;
providing a voicesetting editor interface;
in response to receiving a selection to launch the voicesetting editor interface:
parsing the received textual input and any selected emoji objects;
and
displaying the parsed textual input and any selected emoji objects as selectable tokens;
in response to receiving a selection of a token, displaying a set of prosodic properties that can be applied to the selected token;
and
in response to receiving a selection of a prosodic property, displaying a value associated with the selected prosodic property for allowing a user to adjust the value for controlling expressivity of the textual input when rendered.

15 more elements

A computer readable storage device including computer readable instructions, which when executed by a processing unit is operative to:

display an expressive keyboard, wherein the expressive keyboard includes an alpha-numeric keyboard for receiving textual input and a plurality of emoji objects for selectively applying a vocal sound effect associated with each of the plurality of emoji objects to received textual input;
receive textual input;
display on the expressive keyboard a set of the plurality of emoji objects, based on linguistic properties of the received textual input;
select, by a user, at least one of the displayed plurality of emoji objects;
in response to receiving the selection of the at least one emoji object, identify a predefined vocal sound effect associated with the selected emoji object;
combine the identified vocal sound effect with the received textual input;
output the combined set of identified vocal sound effect and received textual input to a speech generation engine for generating expressive synthesized speech;
providing a voicesetting editor interface;
in response to receiving a selection to launch the voicesetting editor interface:
parsing the received textual input and any selected emoji objects;
and
displaying the parsed textual input and any selected emoji objects as selectable tokens;
in response to receiving a selection of a token, displaying a set of prosodic properties that can be applied to the selected token;
and
in response to receiving a selection of a prosodic property, displaying a value associated with the selected prosodic property for allowing a user to adjust the value for controlling expressivity of the textual input when rendered.

12 more elements

Dependencies are inferred from the claim text.

Facts

Application: 15/347,653Filed 9 Nov 2016
Published: US 2018/0130459 A110 May 2018
Granted: 3 May 2022
Term ends: Not estimatedThe source records don’t support an estimate.

Classification13 codes

G06T 13/00
Animation
G06F 3/0236
Character input methods: using selection techniques to select from displayed items
G06F 3/0237
Character input methods: using prediction or retrieval techniques
G06F 3/04817
Based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like…:…