# US 11,749,256 B2 · Development of voice and other interaction applications

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,749,256 B2
- Document ID: US11749256B2
- Status: Granted
- Kind: B2
- Application number: 16/989,807
- Filed: 2020-08-10
- Published: 2021-02-25
- Granted: 2023-09-05
- Earliest claimed: 2019-08-19
- Assignees: Voicify, LLC
- Applicants: Voicify, LLC
- Inventors: Jeffrey K. McMahon; Robert T. Naughton; Nicholas G. Laidlaw; Alexander M. Dunn; Jason Green
- Web page: https://patentgrep.com/patent/US11749256B2

## CPC

- G10L 13/08: Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination
- G06F 3/04817: using icons (graphical or visual programming using iconic symbols G06F8/34)
- G10L 13/0335: {Pitch control}
- G10L 13/047: Architecture of speech synthesisers
- G10L 13/033: Voice editing, e.g. manipulating the voice of the synthesiser
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Among other things, a developer of an interaction application for an enterprise can create items of content to be provided to an assistant platform for use in responses to requests of end-users. The developer can deploy the interaction application using defined items of content and an available general interaction model including intents and sample utterances having slots. The developer can deploy the interaction application without requiring the developer to formulate any of the intents, sample utterances, or slots of the general interaction model.

## Claims

### Claim 1

A computer-implemented method comprising
presenting a user interface enabling a developer to create speech markup language strings conforming to a speech markup language definition applied by a corresponding interaction assistant platform, wherein the user interface comprises
a text entry area, and
user-entered text in the text entry area;
receiving user input indicating that nested audio effects are to be applied to text portions of the user-entered text when audio corresponding to the text portions is generated by an interaction assistant and
in response to the user input, dynamically adding graphical elements in-line with the user-entered text in the text entry area, the graphical elements indicating the presence of the nested audio effects,
wherein the graphical elements comprise respective backgrounds that are
displayed with the text portions to which the corresponding nested audio effects apply, and
nested in accordance with the nested audio effects.

### Claim 2 (depends on 1)

The method of claim 1 in which the user interface comprises selectable controls corresponding to different types of audio effects.

### Claim 3 (depends on 2)

The method of claim 2 in which the selectable controls comprise a first selectable control that, when selected, causes display of a control for configuring a property of an audio effect corresponding to the first selectable control.

### Claim 4 (depends on 2)

The method of claim 2 in which the selectable controls comprise icons.

### Claim 5 (depends on 1)

The method of claim 1 in which the respective backgrounds are color-coded according to the audio effects to which the respective backgrounds correspond.

### Claim 6 (depends on 1)

The method of claim 1 in which the user interface comprises a selectable control that, when selected, causes display of a raw speech markup language string corresponding to the user-entered text having the nested audio effects.

### Claim 7 (depends on 1)

The method of claim 1, wherein the nested audio effects comprise one or more of speed, pitch, emphasis, or volume.

### Claim 8 (depends on 1)

The method of claim 1, wherein the graphical elements comprise a value indicator that graphically indicates a scale value of at least one of the nested audio effects.

### Claim 9 (depends on 1)

The method of claim 1, wherein the graphical elements are removable by backspacing in-line.

### Claim 10 (depends on 1)

The method of claim 1, wherein a first graphical element of the graphical elements comprises padding indicating that a first nested audio effect, corresponding to the first graphical element, ends at a same position in the user-entered text as a second nested audio effect.

### Claim 11

A computer-implemented method, comprising:
presenting a user interface enabling a developer to create speech markup language strings conforming to a speech markup language definition applied by a corresponding interaction assistant platform, wherein the user interface comprises
a text entry area, and
user-entered text in the text entry area;
receiving user input indicating that audio effects are to be applied to text portions of the user-entered text when audio corresponding to the text portions is generated by an interaction assistant; and
in response to the user input, dynamically adding graphical elements in-line with the user-entered text in the text entry area, the graphical elements indicating the presence of the audio effects,
wherein the graphical elements comprise respective backgrounds that are
displayed with the text portions to which the corresponding audio effects apply, and
color-coded according to the corresponding audio effects.

### Claim 12 (depends on 11)

The method of claim 11 in which the user interface comprises selectable controls corresponding to different types of audio effects.

### Claim 13 (depends on 12)

The method of claim 12 in which the selectable controls comprise a first selectable control that, when selected, causes display of a control for configuring a property of an audio effect corresponding to the first selectable control.

### Claim 14 (depends on 11)

The method of claim 11 in which the user interface comprises a selectable control that, when selected, causes display of a raw speech markup language string corresponding to the user-entered text having the audio effects.

### Claim 15 (depends on 11)

The method of claim 11, wherein the audio effects comprise one or more of speed, pitch, emphasis, or volume.

### Claim 16 (depends on 11)

The method of claim 11, wherein the graphical elements comprise a value indicator that graphically indicates a scale value of at least one of the audio effects.

### Claim 17 (depends on 11)

The method of claim 11, wherein the graphical elements are removable by backspacing in-line.

### Claim 18

One or more non-transitory machine-readable storage devices storing instructions that, upon execution by one or more processing devices, cause the one or more processing devices to perform operations comprising:
presenting a user interface enabling a developer to create speech markup language strings conforming to a speech markup language definition applied by a corresponding interaction assistant platform, wherein the user interface comprises
a text entry area, and
user-entered text in the text entry area;
receiving user input indicating that nested audio effects are to be applied to text portions of the user-entered text when audio corresponding to the text portions is generated by an interaction assistant; and
in response to the user input, dynamically adding graphical elements in-line with the user-entered text in the text entry area, the graphical elements indicating the presence of the nested audio effects,
wherein the graphical elements comprise respective backgrounds that are
displayed with the text portions to which the corresponding nested audio effects apply, and
nested in accordance with the nested audio effects.
