# US 2020/0349932 A1 · Oral communication device and computing systems for processing data and outputting oral feedback, and related methods

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0349932 A1
- Document ID: US20200349932A1
- Status: Published application
- Kind: A1
- Application number: 16/314,744
- Filed: 2018-08-10
- Published: 2020-11-05
- Earliest claimed: 2017-08-10
- Assignees: Facet Labs, LLC
- Applicants: Facet Labs, LLC
- Inventors: STUART OGAWA; LINDSAY ALEXANDER SPARKS; KOICHI NISHIMURA; WILFRED P. SO
- Web page: https://patentgrep.com/patent/US20200349932A1

## CPC

- G10L 13/00: Speech synthesis; Text to speech systems
- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G06F 16/61: Indexing; Data structures therefor; Storage structures
- G06Q 10/00: Administration; Management
- G10L 13/0335: {Pitch control}
- G10L 13/047: Architecture of speech synthesisers
- G10L 13/08: Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination
- G10L 15/05: Word boundary detection
- G10L 15/1815: {Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning}
- G10L 15/30: Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
- H04L 51/02: using automatic reactions or user delegation, e.g. automatic replies or chatbot-generated messages
- G10L 2015/223: {Execution procedure of a spoken command}
- G10L 2015/228: {of application context}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

Typical graphical user interfaces and predefined data fields limit the interaction between a person and a computing system. An oral communication device and a data enablement platform are provided for ingesting oral conversational data from people, and using machine learning to provide intelligence. At the front end, an oral conversational bot, or chatbot, interacts with a user. On the backend, the data enablement platform has a computing architecture that ingests data from various external data sources as well as data from internal applications and databases. These data and algorithms are applied to surface new data, identify trends, provide recommendations, infer new understanding, predict actions and events, and automatically act on this computed information. The chatbot then provides audio data that reflects the information computed by the data enablement platform. The system and the devices, for example, are adaptable to various industries.

## Claims

### Claim 1

A speech computing device comprising a housing that holds at least:
a memory device that stores thereon at least a data enablement application that comprises multiple modules that correspond to different modes, a conversational bot and one or more synthesized voice libraries, wherein each of the one or more synthesized voice libraries comprise voice parameter features of one or more corresponding people;
an input sensor that is configured to record a user's input as speech data;
a processor configured to use the conversational bot to identify contextual data associated with the speech data, the contextual data including a current mode corresponding to a currently activated module of the data enablement application;
a data communication device configured to transmit the audio data and the contextual data via a data network and, in response, receive response data, wherein the response data comprises an indication of a given synthesized voice library, text data, and the current mode;
the processor further configured to use the conversational bot to generate an audio response from the given synthesized voice library and the text data, and to propagate the text data to one or more other modules that are currently inactive; and
an audio speaker that is controlled by the processor to output the audio response.

### Claim 2 (depends on 1)

The computing device of claim 1 wherein the currently activated module is a meeting notes module; the speech data comprises a topic; and the text data comprises data in relation to the topic.

### Claim 3 (depends on 2)

The computing device of claim 2 wherein the computing device detects at least one of a pause or an end of a sentence in the speech data and then outputs the audio response.

### Claim 4 (depends on 2)

The computing device of claim 2 wherein the speech data and the text data is added to a meeting notes file.

### Claim 5 (depends on 2)

The computing device of claim 2 wherein the data communication device is communication with at least one other user device, and the computing device further transmits additional data about the topic to the other user device within a same given time period as outputting the audio response.

### Claim 6 (depends on 2)

The computing device of claim 2 wherein the input sensor obtains public speech data, the computing device further receives private meeting notes, and the computing device further generates meeting notes that comprise the privates meeting notes and public notes derived from the public speech data and the text data in the response data.

### Claim 7 (depends on 6)

The computing device of claim 6 wherein the private notes and the public notes are organized by time of creation.

### Claim 8 (depends on 6)

The computing device of claim 6 wherein the data communication device is in communication with at least a silent communication device to obtain private speech data; and the computing device further generates the private meeting notes from the private speech data.

### Claim 9 (depends on 1)

The computing device of claim 1 further comprising a visual display device, and the response data further includes visual data that is outputted with the audio response.

### Claim 10 (depends on 9)

The computing device of claim 9 wherein the visual display device is a projector.

### Claim 11 (depends on 1)

The computing device of claim 1 wherein the speech data comprises a topic; and the text data comprises a summarization of multiple news articles in relation to the topic.

### Claim 12 (depends on 1)

The computing device of claim 1 wherein the currently activated module is an introductions module associated with a social network platform of the user; the speech data comprises a topic or an entity; and the text data comprises a list of names obtained from the social network platform that are related to the topic or the entity.

### Claim 13 (depends on 1)

The computing device of claim 1 wherein the multiple modules comprise a to-do list module, an opportunities module, an introductions module, a meeting notes module and a new module; and wherein the currently activated of the multiple modules propagates the text data to at least two or more of the other ones of the multiple modules.

### Claim 14 (depends on 1)

The computing device of claim 1 wherein the memory device further stores conversation libraries that include one or more parameters that are used by the conversational bot to affect the audio response; and the parameters comprise one or more of: tone; frequency; loudness; rate at which a word or phrase is said; phonetic pronunciation; lexicon; syntax; articulation; rhythm; melody; phrases; and questions.

### Claim 15 (depends on 1)

The computing device of claim 1 wherein the speech data comprises a topic; and the indication of the given synthesized voice library is associated with a person that is an authority or an expert of the topic.

### Claim 16 (depends on 1)

The computing device of claim 1 wherein the speech data comprises a topic and a name of a person; the indication of the given synthesized voice library is associated with the person; and the text data is in relation to both the topic and the person.

### Claim 17 (depends on 1)

The computing device of claim 1 further comprising a graphics processing unit (GPU) that exchanges data with the processor, the GPU configured to pre-process the audio data using parallel threaded computations to extract data features, and the data communication device transmits the extracted data features in association with the contextual data and the audio data.

### Claim 18

(canceled)
