# US 11,615,791 B2 · Voice application platform

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,615,791 B2
- Document ID: US11615791B2
- Status: Granted
- Kind: B2
- Application number: 16/589,288
- Filed: 2019-10-01
- Published: 2020-08-27
- Granted: 2023-03-28
- Earliest claimed: 2018-06-05
- Assignees: Voicify, LLC
- Applicants: Voicify, LLC
- Inventors: Robert T. Naughton; Nicholas G. Laidlaw; Alexander M. Dunn; Jeffrey K. McMahon
- Web page: https://patentgrep.com/patent/US11615791B2

## CPC

- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L 15/30: Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
- H04M 3/4936: {Speech interaction details (speech recognition per se G10L15/00)}
- H04M 3/4931: {Directory assistance systems}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- H04M: TELEPHONIC COMMUNICATION (circuits for controlling other apparatus via a telephone cable and not involving telephone switching apparatus G08)

## Abstract

Among other things, requests are received from voice assistant devices expressed in accordance with different corresponding protocols of one or more voice assistant frameworks. Each of the requests represents a voiced input by a user to the corresponding voice assistant device. The received requests are re-expressed in accordance with a common request protocol. Based on the received requests, responses to the requests are expressed in accordance with a common response protocol. Each of the responses is re-expressed according to a protocol of the framework with respect to which the corresponding request was expressed. The responses are sent to the voice assistant devices for presentation to the users.

## Claims

### Claim 1

A computer-implemented method comprising:
exposing, through a user interface of a voice application platform, features that enable selecting and managing one or more items of content, wherein the one or more items of content are to be included in phrases spoken to voice assistant devices by end users;
aggregating usage data associated with message elements processed by one or more voice assistant frameworks, wherein the usage data comprises one or more of
success rates of the one or more voice assistant frameworks in invoking features based on the message elements, misses of the one or more voice assistant frameworks in invoking items of content based on the message elements, or information about a request message that includes the message elements;
generating, for at least a first item of content of the one or more items of content, from the aggregated usage data, information about respective performances of the one or more voice assistant frameworks in interpreting one or more words or numbers of the first item of content;
as the one or more items of content are being selected and managed, exposing, through the user interface, the information about the respective performances of the one or more voice assistant frameworks in interpreting the one or more words or numbers of the first item of content;
receiving, through the user interface, inputs indicating selected items of content among the one or more items of content; and
executing a voice application including the selected items of content.

### Claim 2 (depends on 1)

The method of claim 1 in which the usage data is aggregated through a universal API.

### Claim 3 (depends on 1)

The method of claim 1 in which the information about the respective performances is generated by a machine learning algorithm.

### Claim 4 (depends on 1)

The method of claim 1 comprising exposing, through the user interface, information about respective capabilities of types of voice assistant devices in supporting the one or more items of content.

### Claim 5 (depends on 4)

The method of claim 4 in which the information about respective capabilities is based on media formats of the one or more items of content.

### Claim 6 (depends on 1)

The method of claim 1, wherein the information about the respective performances of the one or more voice assistant frameworks in interpreting the one or more words or numbers of the first item of content comprises success rates of the one or more voice assistant frameworks in responding to a request that includes types of words or numbers matching types of the words or numbers of the first item of content.

### Claim 7 (depends on 1)

The method of claim 1, comprising:
identifying, based on the aggregated usage data, that a first voice assistant framework of the one or more voice assistant frameworks is less accurate than another voice assistant framework at interpreting a type of word, a type of number, a sentence structure, or a combination thereof,
wherein the information about the respective performances of the one or more voice assistant frameworks in interpreting the one or more words or numbers of the first item of content is based on the identification that the first voice assistant framework is less accurate than the other voice assistant framework at interpreting the type of word, the type of number, the sentence structure, or the combination thereof.

### Claim 8 (depends on 1)

The method of claim 1, comprising:
exposing, through the user interface, a suggestion for an alteration to the one or more items of content, wherein the suggestion is based on the information about the respective performances of the one or more voice assistant frameworks in interpreting the one or more words or numbers of the first item of content.

### Claim 9 (depends on 8)

The method of claim 8, wherein the suggestion comprises one or more of a rephrasing of a sentence, a removal of a word, an addition of a wording variation, a removal of the wording variation, or an update to a slot value.

### Claim 10

A computer-readable non-transitory storage storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:
exposing, through a user interface of a voice application platform, features that enable selecting and managing one or more items of content, wherein the one or more items of content are to be included in phrases spoken to voice assistant devices by end users;
aggregating usage data associated with message elements processed by one or more voice assistant frameworks, wherein the usage data comprises one or more of
success rates of the one or more voice assistant frameworks in invoking features based on the message elements, misses of the one or more voice assistant frameworks in invoking items of content based on the message elements, or information about a request message that includes the message elements;
generating, for at least a first item of content of the one or more items of content, from the aggregated usage data, information about respective performances of the one or more voice assistant frameworks in interpreting one or more words or numbers of the first item of content;
as the one or more items of content are being selected and managed, exposing, through the user interface, the information about the respective performances of the one or more voice assistant frameworks in interpreting the one or more words or numbers of the first item of content; receiving, through the user interface, inputs indicating selected items of content among the one or more items of content; and
executing a voice application including the selected items of content.

### Claim 11 (depends on 10)

The non-transitory storage of claim 10 in which the usage data is aggregated through a universal API.

### Claim 12 (depends on 11)

The non-transitory storage of claim 11 in which the information about the respective performances is generated by a machine learning algorithm.

### Claim 13 (depends on 10)

The non-transitory storage of claim 10, wherein the operations comprise exposing, through the user interface, information about respective capabilities of types of voice assistant devices in supporting the one or more items of content.

### Claim 14 (depends on 13)

The non-transitory storage of claim 13 in which the information about respective capabilities is based on media formats of the one or more items of content.

### Claim 15 (depends on 10)

The non-transitory storage of claim 10, wherein the information about the respective performances of the one or more voice assistant frameworks in interpreting the one or more words or numbers of the first item of content comprises success rates of the one or more voice assistant frameworks in responding to a request that includes types of words or numbers matching types of the words or numbers of the first item of content.

### Claim 16 (depends on 10)

The non-transitory storage of claim 10, wherein the operations comprise:
identifying, based on the aggregated usage data, that a first voice assistant framework of the one or more voice assistant frameworks is less accurate than another voice assistant framework at interpreting parsing a type of word, a type of number, a sentence structure, a type of data, or a combination thereof,
wherein the information about the respective performances of the one or more voice assistant frameworks in interpreting the one or more words or numbers of parsing the first item of content is based on the identification that the first voice assistant framework is less accurate than the other voice assistant framework at interpreting parsing the type of word, the type of number, the sentence structure, the type of data, or the combination thereof.

### Claim 17 (depends on 10)

The non-transitory storage of claim 10, wherein the operations comprise:
exposing, through the user interface, a suggestion for an alteration to the one or more items of content, wherein the suggestion is based on the information about the respective performances of the one or more voice assistant frameworks in interpreting the one or more words or numbers of the first item of content.

### Claim 18

A system comprising:
a processor; and
a non-transitory storage storing instructions that, when executed by the processor, cause the processor to perform operations comprising:
exposing, through a user interface of a voice application platform, features that enable selecting and managing one or more items of content, wherein the one or more items of content are to be included in phrases spoken to voice assistant devices by end users;
aggregating usage data associated with message elements processed by one or more voice assistant frameworks, wherein the usage data comprises one or more of
success rates of the one or more voice assistant frameworks in invoking features based on the message elements, misses of the one or more voice assistant frameworks in invoking items of content based on the message elements, or information about a request message that includes the message elements;
generating, for at least a first item of content of the one or more items of content, from the aggregated usage data, information about respective performances of the one or more voice assistant frameworks in interpreting one or more words or numbers of the first item of content;
as the one or more items of content are being selected and managed, exposing, through the user interface, the information about the respective performances of the one or more voice assistant frameworks in interpreting the one or more words or numbers of the first item of content; receiving, through the user interface, inputs indicating selected items of content among the one or more items of content; and
executing a voice application including the selected items of content.

### Claim 19

A computer-implemented method comprising:
exposing, through a user interface of a voice application platform, features that enable selecting and managing one or more items of content, wherein the one or more items of content are to be included in phrases spoken to voice assistant devices by end users;
determining, for at least a first item of content of the one or more items of content, information about respective performances of one or more voice assistant frameworks in interpreting one or more words or numbers of the first item of content,
wherein the information about the respective performances of the one or more voice assistant frameworks in interpreting the one or more words or numbers of the first item of content comprises success rates of the one or more voice assistant frameworks in responding to a request that includes types of words or numbers matching types of the words or numbers of the first item of content;
as the one or more items of content are being selected and managed, exposing, through the user interface, the information about the respective performances of the one or more voice assistant frameworks in interpreting the one or more words or numbers of the first item of content;
receiving, through the user interface, inputs indicating selected items of content among the one or more items of content; and
executing a voice application including the selected items of content.

### Claim 20 (depends on 19)

The method of claim 19, comprising:
exposing, through the user interface, a suggestion for an alteration to the one or more items of content, wherein the suggestion is based on the information about the respective performances of the one or more voice assistant frameworks in interpreting the one or more words or numbers of the first item of content.

### Claim 21 (depends on 20)

The method of claim 20, wherein the suggestion comprises one or more of a rephrasing of a sentence, a removal of a word, an addition of a wording variation, a removal of the wording variation, or an update to a slot value.

### Claim 22

A computer-readable non-transitory storage storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:
exposing, through a user interface of a voice application platform, features that enable selecting and managing one or more items of content, wherein the one or more items of content are to be included in phrases spoken to voice assistant devices by end users;
determining, for at least a first item of content of the one or more items of content, information about respective performances of one or more voice assistant frameworks in interpreting one or more words or numbers of the first item of content,
wherein the information about the respective performances of the one or more voice assistant frameworks in interpreting the one or more words or numbers of the first item of content comprises success rates of the one or more voice assistant frameworks in responding to a request that includes types of words or numbers matching types of the words or numbers of the first item of content;
as the one or more items of content are being selected and managed, exposing, through the user interface, the information about the respective performances of the one or more voice assistant frameworks in interpreting the one or more words or numbers of the first item of content;
receiving, through the user interface, inputs indicating selected items of content among the one or more items of content; and
executing a voice application including the selected items of content.

### Claim 23 (depends on 22)

The non-transitory storage of claim 22, wherein the operations comprise:
exposing, through the user interface, a suggestion for an alteration to the one or more items of content, wherein the suggestion is based on the information about the respective performances of the one or more voice assistant frameworks in interpreting the one or more words or numbers of the first item of content.

### Claim 24 (depends on 23)

The non-transitory storage of claim 23, wherein the suggestion comprises one or more of a rephrasing of a sentence, a removal of a word, an addition of a wording variation, a removal of the wording variation, or an update to a slot value.

### Claim 25

A system comprising:
a processor; and
a non-transitory storage storing instructions that, when executed by the processor, cause the processor to perform operations comprising:
exposing, through a user interface of a voice application platform, features that enable selecting and managing one or more items of content, wherein the one or more items of content are to be included in phrases spoken to voice assistant devices by end users;
determining, for at least a first item of content of the one or more items of content, information about respective performances of one or more voice assistant frameworks in interpreting one or more words or numbers of the first item of content,
wherein the information about the respective performances of the one or more voice assistant frameworks in interpreting the one or more words or numbers of the first item of content comprises success rates of the one or more voice assistant frameworks in responding to a request that includes types of words or numbers matching types of the words or numbers of the first item of content;
as the one or more items of content are being selected and managed, exposing, through the user interface, the information about the respective performances of the one or more voice assistant frameworks in interpreting the one or more words or numbers of the first item of content;
receiving, through the user interface, inputs indicating selected items of content among the one or more items of content; and
executing a voice application including the selected items of content.

### Claim 26 (depends on 25)

The system of claim 25, wherein the operations comprise:
exposing, through the user interface, a suggestion for an alteration to the one or more items of content, wherein the suggestion is based on the information about the respective performances of the one or more voice assistant frameworks in interpreting the one or more words or numbers of the first item of content.

### Claim 27 (depends on 26)

The system of claim 26, wherein the suggestion comprises one or more of a rephrasing of a sentence, a removal of a word, an addition of a wording variation, a removal of the wording variation, or an update to a slot value.
