# US 2022/0030312 A1 · Speech interface

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0030312 A1
- Document ID: US20220030312A1
- Status: Published application
- Kind: A1
- Application number: 17/495,564
- Filed: 2021-10-06
- Published: 2022-01-27
- Earliest claimed: 2001-10-03
- Assignees: Promptu Systems Corporation
- Applicants: Promptu Systems Corporation
- Inventors: Adam Jordan; Scott Lynn Maddux; Tim Plowman; Victoria Stanbach; Jody Williams
- Web page: https://patentgrep.com/patent/US20220030312A1

## CPC

- H04N 21/47: End-user applications
- G06F 3/16: Sound input; Sound output (speech processing G10L)
- G06Q 30/0271: {Personalized advertisement}
- G06Q 30/0631: {Recommending goods or services}
- G10L 13/00: Speech synthesis; Text to speech systems
- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L 21/06: Transformation of speech into a non-audible representation, e.g. speech visualisation or speech processing for tactile aids (G10L15/26 takes precedence)
- H04N 21/42203: {sound input device, e.g. microphone}
- H04N 21/4221: {Dedicated function buttons, e.g. for the control of an EPG, subtitles, aspect ratio, picture-in-picture or teletext}
- H04N 21/4316: {for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window}
- H04N 21/4622: {Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet (web site content organization and management for information retrieval from the Internet G06F16/958; transmission by internet of broadcast information H04H60/82; stock exchange data over packet-switching network H04L12/1804; push services including data channel over packet-switching network H04L12/1859)}
- H04N 21/472: End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content {(end-user interfaces for retrieving video data from a database G06F16/739; network services for supporting unicast streaming H04L65/612)}
- H04N 21/47202: {for requesting content on demand, e.g. video on demand}
- H04N 21/47211: {for requesting pay-per-view content (payment schemes payment architectures or payment protocols G06Q20/00, G07F)}
- H04N 21/47214: {for content reservation or setting reminders; for requesting event notification, e.g. of sport results or stock market (stock exchange data over packet-switching network H04L12/1804; push services over packet-switching network H04L12/1859; notification of incoming messages in packet switching networks H04L51/224)}
- H04N 21/475: End-user interface for inputting end-user data, e.g. personal identification number [PIN], preference data
- H04N 21/478: Supplemental services, e.g. displaying phone caller identification, shopping application
- H04N 21/4781: {Games}
- H04N 21/4782: Web browsing {, e.g. WebTV}
- H04N 21/4788: communicating with other users, e.g. chatting {(arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104)}
- H04N 21/482: End-user interface for programme selection
- H04N 21/4826: {using recommendation lists, e.g. of programmes or channels sorted out according to their score}
- H04N 21/4828: {for searching programme descriptors (retrieval of video data G06F16/739)}
- H04N 21/4852: {for modifying audio parameters, e.g. switching between mono and stereo}
- H04N 21/812: {involving advertisement data (advertising per se G06Q30/02)}
- H04N 21/8173: {End-user applications, e.g. Web browser, game}
- G10L 2015/221: {Announcement of recognition results}
- G10L 2015/223: {Execution procedure of a spoken command}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

A system (100) for enabling a user to select media content in an entertainment environment, comprising a remote control device (110) having a set of user-activated keys and a speech activation circuit adapted to enable a speech signal; a speech engine (160) comprising a speech recognizer (170); an application wrapper (180) configured to recognize substantive meaning in the speech signal; and a media content controller (190) configured to select media content. Every function that can be executed by activation of the user-activated keys can also be executed by the speech engine (160) in response to the recognized substantive meaning.

## Claims

### Claim 1

A system for enabling a user to select media content in an entertainment environment, said system comprising:
a remote control device comprising:
a set of user activated keys,
a microphone for receiving user speech, and
a transmitter configured to transmit a speech signal;
a speech engine configured to:
receive the speech signal,
provide feedback information comprising a list of possible matches associated with the speech signal, and
output commands for enabling selection of media content; and
a media content controller configured to: receive the outputted commands, select media content, and use selected media content.

### Claim 2 (depends on 1)

The system of claim 1, wherein every function of the entertainment environment that can be executed by activation of the user activated keys can also be executed by the speech engine.

### Claim 3 (depends on 1)

The system of claim 1, wherein the speech engine is further configured to transcribe the speech signal into textual information.

### Claim 4 (depends on 3)

The system of claim 3, wherein the textual information is represented by binary streams.

### Claim 5 (depends on 1)

The system of claim 1, wherein the transmitter is configured to forward the speech signal from the remote control device to the speech engine without modifying the speech signal.

### Claim 6 (depends on 1)

The system of claim 1, wherein:
the speech engine is further configured to recognize meaning embodied in the speech signal; and
the feedback information denotes whether substantive meaning has been successfully recognized in the speech signal.

### Claim 7 (depends on 6)

The system of claim 6, wherein recognizing substantive meaning embodied in the speech signal comprises generating a set of navigation grammars and a vocabulary.

### Claim 8 (depends on 6)

The system of claim 6, wherein the feedback information comprises an indication of unsuccessful recognition.

### Claim 9 (depends on 6)

The system of claim 6, further comprising a display, and further wherein:
the system is configured to enable text on the display to be selected by activation of the user activated keys; and
the system is further configured to enable text on the display to be selected by the speech engine in response to the recognized substantive meaning.

### Claim 10 (depends on 1)

The system of claim 1, wherein the speech engine is further configured to provide the feedback information to a set-top-box for visual display to a user.

### Claim 11 (depends on 10)

The system of claim 10, wherein the feedback information comprises an indication that:
a push-to-talk button is pressed down, or
that the system is listening to or processing an utterance.

### Claim 12 (depends on 10)

The system of claim 10, wherein the feedback information comprises an indication of unsuccessful recognition selected from one of a plurality of levels.

### Claim 13 (depends on 1)

The system of claim 1, wherein the speech engine is further configured to:
receive, after receiving a portion of the speech signal but before receiving all of the speech signal, a signal indicating activation of one of the set of user activated keys;
cancel the output of a command associated with the speech signal.

### Claim 14 (depends on 1)

The system of claim 1, wherein the speech engine is further configured to:
receive, after receiving the speech signal but before processing the list of possible matches, a signal indicating activation of one of the set of user activated keys;
cancel the output of a command associated with the speech signal.

### Claim 15 (depends on 1)

The system of claim 1, wherein the speech engine is further configured to:
receive, after receiving the speech signal but before outputting a command associated with the speech signal, a signal indicating activation of one of the set of user activated keys;
determine, based on a sequence of the speech signal and the activation signal, whether to:
cancel the output of a command associated with the speech signal; or
output the command associated with the speech signal and a command associated with the activation signal.

### Claim 16 (depends on 1)

The system of claim 1, wherein the speech engine is further configured to:
receive, after receiving the speech signal but before outputting a command associated with the speech signal, a second speech signal;
determine, based on a sequence of the speech signal and the second speech signal, whether to:
cancel the output of a command associated with the speech signal; or
output the command associated with the speech signal and a command associated with the second speech signal.

### Claim 17 (depends on 1)

The system of claim 1, wherein the feedback information comprises at least one overlay.

### Claim 18 (depends on 1)

The system of claim 1, wherein the set of user activated keys are adapted to execute functions of the entertainment environment.

### Claim 19

A method for enabling a user to select media content in an entertainment environment, comprising:
receiving a speech signal from a remote control device;
providing feedback information comprising a list of possible matches associated with the speech signal;
outputting commands for enabling selection of media content; and
selecting and using media content based on the outputted commands.

### Claim 20 (depends on 19)

The method of claim 19, further comprising transcribing the speech signal into textual information.

### Claim 21 (depends on 20)

The method of claim 20, wherein the textual information is represented by binary streams.

### Claim 22 (depends on 19)

The method of claim 19, further comprising forwarding the speech signal to a speech engine without modifying the speech signal.

### Claim 23 (depends on 19)

The method of claim 19, further comprising recognizing meaning embodied in the speech signal; and, further wherein the feedback information denotes whether substantive meaning has been successfully recognized in the speech signal.

### Claim 24 (depends on 23)

The method of claim 23, wherein recognizing substantive meaning embodied in the speech signal comprises generating a set of navigation grammars and a vocabulary.

### Claim 25 (depends on 23)

The method of claim 23, wherein the feedback information comprises an indication of unsuccessful recognition.

### Claim 26 (depends on 23)

The method of claim 23, further comprising:
enabling displayed text to be selected using the remote control device; and
enabling displayed text to be selected in response to the recognized substantive meaning.

### Claim 27 (depends on 19)

The method of claim 19, further comprising providing the feedback information to a set-top-box for visual display to a user.

### Claim 28 (depends on 27)

The method of claim 27, wherein the feedback information comprises an indication that:
a push-to-talk button is pressed down, or
an utterance is being listened to or processed.

### Claim 29 (depends on 27)

The method of claim 27, wherein the feedback information comprises an indication of unsuccessful recognition selected from one of a plurality of levels.

### Claim 30 (depends on 19)

The method of claim 19, further comprising:
receiving, after receiving a portion of the speech signal but before receiving all of the speech signal, a signal indicating activation of a user activated key on the remote control device; and
canceling the output of a command associated with the speech signal.

### Claim 31 (depends on 19)

The method of claim 19, further comprising:
receiving, after receiving the speech signal but before processing the list of possible matches, a signal indicating activation of a user activated key on the remote control device; and
canceling the output of a command associated with the speech signal.

### Claim 32 (depends on 19)

The method of claim 19, further comprising:
receiving, after receiving the speech signal but before outputting a command associated with the speech signal, a signal indicating activation of a user activated key on the remote control device;
determining, based on a sequence of the speech signal and the activation signal, whether to:
cancel the output of a command associated with the speech signal; or
output the command associated with the speech signal and a command associated with the activation signal.

### Claim 33 (depends on 19)

The method of claim 19, further comprising:
receive, after receiving the speech signal but before outputting a command associated with the speech signal, a second speech signal;
determine, based on a sequence of the speech signal and the second speech signal, whether to:
cancel the output of a command associated with the speech signal; or
output the command associated with the speech signal and a command associated with the second speech signal.

### Claim 34 (depends on 19)

The method of claim 19, wherein the feedback information comprises at least one overlay.

### Claim 35 (depends on 19)

The method of claim 19, wherein the remote control device comprises a set of user activated keys adapted to execute functions of the entertainment environment.

### Claim 36

A system for enabling a user to select media content in an entertainment environment, said system comprising:
a first means for receiving user speech and transmitting a speech signal;
a second means for:
receiving the speech signal from the first means,
providing feedback information comprising a list of possible matches associated with the speech signal,
outputting commands for enabling selection of media content, and
selecting and using media content based on the outputted commands.

### Claim 37 (depends on 36)

The system of claim 36, wherein every function of the entertainment environment that can be executed by activation of user activated keys on the first means can also be executed by the second means.

### Claim 38 (depends on 36)

The system of claim 36, wherein the second means is further configured to transcribe the speech signal into textual information.

### Claim 39 (depends on 38)

The system of claim 38, wherein the textual information is represented by binary streams.

### Claim 40 (depends on 36)

The system of claim 36, wherein the first means is configured to forward the speech signal to the second means without modifying the speech signal.

### Claim 41 (depends on 36)

The system of claim 36, wherein every function that can be executed by activation of user activated keys on the first means can also be executed by the second means.

### Claim 42 (depends on 36)

The system of claim 36, wherein:
the second means is further configured to recognize meaning embodied in the speech signal; and
the feedback information denotes whether substantive meaning has been successfully recognized in the speech signal.

### Claim 43 (depends on 42)

The system of claim 42, wherein recognizing substantive meaning embodied in the speech signal comprises generating a set of navigation grammars and a vocabulary.

### Claim 44 (depends on 42)

The system of claim 42, wherein the feedback information comprises an indication of unsuccessful recognition.

### Claim 45 (depends on 42)

The system of claim 42, further comprising a display, and further wherein:
the system is configured to enable text on the display to be selected by activation of user activated keys on the first means; and
the system is further configured to enable text on the display to be selected by the second means in response to the recognized substantive meaning.

### Claim 46 (depends on 36)

The system of claim 36, wherein the second means is further configured to provide the feedback information to a set-top-box for visual display to a user.

### Claim 47 (depends on 46)

The system of claim 46, wherein the feedback information comprises an indication that:
a push-to-talk button is pressed down, or
that the system is listening to or processing an utterance.

### Claim 48 (depends on 46)

The system of claim 46, wherein the feedback information comprises an indication of unsuccessful recognition selected from one of a plurality of levels.

### Claim 49 (depends on 36)

The system of claim 36, wherein the second means is further configured to:
receive, after receiving a portion of the speech signal but before receiving all of the speech signal, a signal indicating activation of a user activated key on the first means; and
cancel the output of a command associated with the speech signal.

### Claim 50 (depends on 36)

The system of claim 36, wherein the second means is further configured to:
receive, after receiving the speech signal but before processing the list of possible matches, a signal indicating activation of a user activated key on the first means; and
cancel the output of a command associated with the speech signal.

### Claim 51 (depends on 36)

The system of claim 36, wherein the second means is further configured to:
receive, after receiving the speech signal but before outputting a command associated with the speech signal, a signal indicating activation of a user activated key on the first means; and
determine, based on a sequence of the speech signal and the activation signal, whether to:
cancel the output of a command associated with the speech signal; or
output the command associated with the speech signal and a command associated with the activation signal.

### Claim 52 (depends on 36)

The system of claim 36, wherein the second means is further configured to:
receive, after receiving the speech signal but before outputting a command associated with the speech signal, a second speech signal; and
determine, based on a sequence of the speech signal and the second speech signal, whether to:
cancel the output of a command associated with the speech signal; or
output the command associated with the speech signal and a command associated with the second speech signal.

### Claim 53 (depends on 36)

The system of claim 36, wherein the feedback information comprises at least one overlay.

### Claim 54 (depends on 36)

The system of claim 36, wherein the first means comprises a set of user activated keys adapted to execute functions of the entertainment environment.
