# US 2023/0066552 A1 · Intelligent automated assistant for tv user interactions

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0066552 A1
- Document ID: US20230066552A1
- Status: Published application
- Kind: A1
- Application number: 17/973,398
- Filed: 2022-10-25
- Published: 2023-03-02
- Earliest claimed: 2014-06-30
- Assignees: Apple Inc.
- Applicants: Apple Inc.
- Inventors: Marcel VAN OS; Harry J. SADDLER; Lia T. NAPOLITANO; Jonathan H. RUSSELL; Patrick M. LISTER; Rohit DASARI
- Web page: https://patentgrep.com/patent/US20230066552A1

## CPC

- H04N 21/4394: {involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams (arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58)}
- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G06F 16/73: Querying
- G06F 16/738: Presentation of query results
- G06F 16/7867: {using information manually generated, e.g. tags, keywords, comments, title and artist information, manually generated time, location and usage information, user ratings}
- G09C 1/00: Apparatus or methods whereby a given sequence of signs, e.g. an intelligible text, is transformed into an unintelligible sequence of signs by transposing the signs or groups of signs or by replacing them by others according to a predetermined system (cryptographic typewriters G09C3/00)
- G10L 15/1815: {Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning}
- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L 15/26: Speech to text systems (G10L15/08 takes precedence)
- G10L 17/00: Speaker identification or verification techniques
- G10L 17/22: Interactive procedures; Man-machine interfaces
- H04N 21/2541: {Rights Management (protecting software against unauthorised usage in a vending or licensing environment G06F21/10; security in data switching network management H04L41/28; security management or policies for network security H04L63/20; access security in wireless networks H04W12/08)}
- H04N 21/25891: {being end-user preferences (retrieval of video data in a video database based on user preferences G06F16/739; arrangements for recognizing users' preferences H04H60/46; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18)}
- H04N 21/4122: {additional display device, e.g. video projector (digital output for controlling a plurality of local displays G06F3/1423)}
- H04N 21/41265: {having a remote control device for bidirectional communication between the remote control device and client device (unidirectional remote control device H04N21/42204)}
- H04N 21/42203: {sound input device, e.g. microphone}
- H04N 21/42222: {Additional components integrated in the remote control device, e.g. timer, speaker, sensors for detecting position, direction or movement of the remote control, microphone or battery charging device}
- H04N 21/4312: {involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations}
- H04N 21/4316: {for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window}
- H04N 21/4438: {Window management, e.g. event handling following interaction with the user interface}
- H04N 21/4532: {involving end-user characteristics, e.g. viewer profile, preferences (monitoring of user activities for profile generation for accessing a video database G06F16/739; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18)}
- H04N 21/4627: Rights management {associated to the content (security in data switching network management H04L41/28; security management or policies for network security H04L63/20; access security in wireless networks H04W12/08)}
- H04N 21/4722: for requesting additional data associated with the content
- H04N 21/482: End-user interface for programme selection
- H04N 21/4826: {using recommendation lists, e.g. of programmes or channels sorted out according to their score}
- H04N 21/4828: {for searching programme descriptors (retrieval of video data G06F16/739)}
- H04N 21/4882: {for displaying messages, e.g. warnings, reminders (arrangements for providing short real-time information to substation in data switching networks H04L12/1895)}
- H04N 21/84: Generation or processing of descriptive data, e.g. content descriptors {(systems specially adapted for using meta-information in broadcast systems H04H60/73)}
- H04N 21/858: Linking data to content, e.g. by linking an URL to a video object, by creating a hotspot
- G10L 2015/223: {Execution procedure of a spoken command}
- H04N 21/47214: {for content reservation or setting reminders; for requesting event notification, e.g. of sport results or stock market (stock exchange data over packet-switching network H04L12/1804; push services over packet-switching network H04L12/1859; notification of incoming messages in packet switching networks H04L51/224)}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G09C: CIPHERING OR DECIPHERING APPARATUS FOR CRYPTOGRAPHIC OR OTHER PURPOSES INVOLVING THE NEED FOR SECRECY
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

Systems and processes are disclosed for controlling television user interactions using a virtual assistant. In an example process, a virtual assistant can interact with a television set-top box to control content shown on a television display. Speech input for the virtual assistant can be received from a device with a microphone. The speech input can comprise a query associated with content shown on the television display. A user intent of the query can be determined based on one or more of the content shown on the television display and a viewing history of media content. A result of the query can be caused to be displayed based on the determined user intent.

## Claims

### Claim 1

An electronic device, comprising:
one or more processors;
a memory; and
one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for:
displaying media content on a display;
receiving an input from a user;
determining one or more virtual assistant queries based on one or more of the media content and a viewing history of media content; and
displaying the one or more virtual assistant queries on the display.

### Claim 2 (depends on 1)

The electronic device of claim 1, wherein the input is received from the user on a remote control.

### Claim 3 (depends on 1)

The electronic device of claim 1, wherein the input is received from the user on a mobile telephone.

### Claim 4 (depends on 1)

The electronic device of claim 1, wherein the one or more virtual assistant queries are overlaid on a moving image.

### Claim 5 (depends on 1)

The electronic device of claim 1, wherein the media content comprises a movie.

### Claim 6 (depends on 1)

The electronic device of claim 1, wherein the one or more virtual assistant queries comprise a query about a person appearing in the media content.

### Claim 7 (depends on 1)

The electronic device of claim 1, wherein the one or more virtual assistant queries comprise a query about a character appearing in the media content.

### Claim 8 (depends on 1)

The electronic device of claim 1, wherein the one or more virtual assistant queries comprise a query for media content associated with a person appearing in the media content.

### Claim 9 (depends on 1)

The electronic device of claim 1, wherein the media content or the viewing history of media content comprise an episode of a television show, and the one or more virtual assistant queries comprise a query about another episode of the television show.

### Claim 10 (depends on 1)

The electronic device of claim 1, wherein the media content or the viewing history of media content comprise an episode of a television show, and the one or more virtual assistant queries comprise a request to set a reminder to watch or record a subsequent episode of the media content.

### Claim 11 (depends on 1)

The electronic device of claim 1, wherein the one or more virtual assistant queries comprise a query for descriptive details of the media content.

### Claim 12 (depends on 11)

The electronic device of claim 11, wherein the descriptive details comprise one or more of a show title, a character list, an actor list, an episode description, a team roster, a team ranking, or a show synopsis.

### Claim 13 (depends on 1)

The electronic device of claim 1, the one or more programs including instructions for:
receiving a selection of one of the one or more virtual assistant queries; and
displaying a result of the selected one of the one or more virtual assistant queries.

### Claim 14 (depends on 1)

The electronic device of claim 1, wherein determining the one or more virtual assistant queries comprises determining the one or more virtual assistant queries based on one or more of a query history, a user preference, or a query popularity.

### Claim 15 (depends on 1)

The electronic device of claim 1, wherein determining the one or more virtual assistant queries comprises determining the one or more virtual assistant queries based on media content available to the user for viewing.

### Claim 16 (depends on 1)

The electronic device of claim 1, wherein determining the one or more virtual assistant queries comprises determining the one or more virtual assistant queries based on a received notification.

### Claim 17 (depends on 1)

The electronic device of claim 1, wherein determining the one or more virtual assistant queries comprises determining the one or more virtual assistant queries based on an active application.

### Claim 18 (depends on 1)

The electronic device of claim 1, the one or more programs including instructions for:
determining alternative media content for display based on one or more of a user preference, a show popularity, and a status of a live sporting event; and
displaying a notification comprising the determined alternative media content.

### Claim 19

A computer-implemented method, comprising:
at an electronic device with one or more processors and memory:
displaying media content on a display;
receiving an input from a user;
determining one or more virtual assistant queries based on one or more of the media content and a viewing history of media content; and
displaying the one or more virtual assistant queries on the display.

### Claim 20

A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors of an electronic device, cause the electronic device to:
display media content on a display;
receive an input from a user;
determine one or more virtual assistant queries based on one or more of the media content and a viewing history of media content; and
display the one or more virtual assistant queries on the display.
