# US 2025/0165130 A1 · Intelligent automated assistant in a media environment

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0165130 A1
- Document ID: US20250165130A1
- Status: Published application
- Kind: A1
- Application number: 19/030,238
- Filed: 2025-01-17
- Published: 2025-05-22
- Earliest claimed: 2015-09-08
- Assignees: Apple Inc.
- Applicants: Apple Inc.
- Inventors: Lia T. NAPOLITANO; Grace H. HWANG; Henrique D. PENHA; Jeremiah D. SHAW; Jorge S. FINO
- Web page: https://patentgrep.com/patent/US20250165130A1

## CPC

- G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence)
- G06F 3/0482: Interaction with lists of selectable items, e.g. menus
- G06F 3/0488: using a touch-screen or digitiser, e.g. input of commands through traced gestures
- G06F 3/165: {Management of the audio stream, e.g. setting of volume, audio stream path}
- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G06F 16/3329: Natural language query formulation
- G06F 16/438: Presentation of query results
- G06F 16/45: Clustering; Classification
- G06F 16/48: Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F 16/483: using metadata automatically derived from the content
- G06F 16/487: using geographical or spatial information, e.g. location
- G06F 16/489: {using time information}
- G06F 16/90332: {Natural language query formulation or dialogue systems}
- G06F 40/40: Processing or translation of natural language (natural language analysis G06F40/20; semantic analysis G06F40/30)
- H04N 21/41265: {having a remote control device for bidirectional communication between the remote control device and client device (unidirectional remote control device H04N21/42204)}
- H04N 21/42203: {sound input device, e.g. microphone}
- H04N 21/42204: {User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor (constructive details of casings for the remote control device H01H9/0235; user interfaces for controlling a tuning device of a television receiver through a remote control H03J9/00; remote control of peripheral devices connected to a television receiver through the remote control device of the television receiver H04B1/205)}
- H04N 21/47: End-user applications
- H04N 21/4828: {for searching programme descriptors (retrieval of video data G06F16/739)}
- G10L 15/00: Speech recognition (G10L17/00 takes precedence)
- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L 15/26: Speech to text systems (G10L15/08 takes precedence)
- G10L 21/0208: Noise filtering
- G10L 2015/221: {Announcement of recognition results}
- G10L 2015/223: {Execution procedure of a spoken command}
- H04N 21/42206: {characterized by hardware details}
- H04N 21/42224: {Touch pad or touch panel provided on the remote control}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

Systems and processes are disclosed for operating a digital assistant in a media environment. In an exemplary embodiment, a user can interact with a digital assistant of a media device while content is displayed by the media device. In one approach, a plurality of exemplary natural language requests can be displayed in response to detecting a user input of a first input type. The plurality of exemplary natural language requests can be contextually-related to the displayed content. In another approach, a user request can be received in response to detecting a user input of a second input type. A task that at least partially satisfies the user request can be performed. The performed task can depend on the nature of the user request and the content being displayed by the media device. In particular, the user request can be satisfied while reducing disruption to user consumption of media content.

## Claims

### Claim 1

An electronic device, comprising:
one or more processors;
a memory; and
one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for:
displaying content on a display unit;
while displaying the content, detecting a user input; and
in response to detecting the user input:
displaying a user interface on the display unit, the user interface including a plurality of exemplary natural language requests that are contextually-related to the displayed content, wherein receiving a user utterance corresponding to one of the plurality of exemplary natural language requests causes the digital assistant to perform a respective action.

### Claim 2 (depends on 1)

The electronic device of claim 1, wherein the displayed content comprises media content.

### Claim 3 (depends on 2)

The electronic device of claim 2, wherein the plurality of exemplary natural language requests includes natural language requests to modify one or more settings associated with the media content.

### Claim 4 (depends on 2)

The electronic device of claim 2, wherein the media content continues to play while the user interface is displayed.

### Claim 5 (depends on 2)

The electronic device of claim 2, the one or more programs including instructions for:
outputting audio associated with the media content, wherein an amplitude of the audio is not reduced in response to detecting the user input.

### Claim 6 (depends on 1)

The electronic device of claim 1, wherein the displayed content comprises a main menu user interface.

### Claim 7 (depends on 6)

The electronic device of claim 6, wherein the plurality of exemplary natural language requests includes exemplary natural language requests related to each of a plurality of core competencies of the digital assistant.

### Claim 8 (depends on 1)

The electronic device of claim 1, wherein the displayed content comprises a second user interface with results associated with a previous user request.

### Claim 9 (depends on 8)

The electronic device of claim 8, wherein the plurality of exemplary natural language requests includes natural language requests to refine the results.

### Claim 10 (depends on 1)

The electronic device of claim 1, wherein the user interface includes textual instructions for invoking and interacting with the digital assistant.

### Claim 11 (depends on 1)

The electronic device of claim 1, wherein the user interface includes a visual indicator indicating that the digital assistant is not receiving audio input.

### Claim 12 (depends on 1)

The electronic device of claim 1, wherein the user interface is overlaid on the displayed content.

### Claim 13 (depends on 1)

The electronic device of claim 1, the one or more programs including instructions for:
in response to detecting the user input, reducing a brightness of the displayed content to highlight the user interface.

### Claim 14 (depends on 1)

The electronic device of claim 1, wherein the user input is detected on a remote control of the electronic device.

### Claim 15 (depends on 14)

The electronic device of claim 14, wherein the user input comprises pressing a button of the remote control device and releasing the button within a predetermined duration after pressing the button.

### Claim 16 (depends on 15)

The electronic device of claim 15, wherein the button is configured to invoke the digital assistant.

### Claim 17 (depends on 1)

The electronic device of claim 1, wherein the user interface includes textual instructions for displaying a virtual keyboard interface.

### Claim 18 (depends on 1)

The electronic device of claim 1, the one or more programs including instructions for:
after displaying the user interface, detecting a second user input; and
in response to detecting the second user input, displaying a virtual keyboard interface on the display unit.

### Claim 19 (depends on 18)

The electronic device of claim 18, the one or more programs including instructions for:
changing a focus of the user interface to a search field on the user interface.

### Claim 20 (depends on 19)

The electronic device of claim 19, wherein the search field is configured to receive text search queries via the virtual keyboard interface.

### Claim 21 (depends on 18)

The electronic device of claim 18, wherein the virtual keyboard interface cannot be used to interact with the digital assistant.

### Claim 22 (depends on 1)

The electronic device of claim 1, wherein the second user input comprises a predetermined motion pattern on a touch-sensitive surface of a remote control device of the electronic device.

### Claim 23 (depends on 1)

The electronic device of claim 1, wherein the plurality of exemplary natural language requests are display at a predetermined amount of time after detecting the user input.

### Claim 24 (depends on 1)

The electronic device of claim 1, wherein displaying the plurality of exemplary natural language requests further comprises:
displaying each of the plurality of exemplary natural language requests one at a time in a predetermined sequence.

### Claim 25 (depends on 24)

The electronic device of claim 24, wherein the sequentially displaying further comprises:
replacing display of a previously displayed exemplary natural language request of the plurality of exemplary natural language requests with a subsequent exemplary natural language request of the plurality of exemplary natural language requests.

### Claim 26 (depends on 1)

The electronic device of claim 1, wherein the content comprises a second user interface with one or more items, wherein a focus of the second user interface is on an item of the one or more items when the user input is detected, and wherein the plurality of exemplary natural language requests are contextually-related to the item of the one or more items.

### Claim 27

A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device, the one or more programs including instructions for:
displaying content on a display unit;
while displaying the content, detecting a user input; and
in response to detecting the user input:
displaying a user interface on the display unit, the user interface including a plurality of exemplary natural language requests that are contextually-related to the displayed content, wherein receiving a user utterance corresponding to one of the plurality of exemplary natural language requests causes the digital assistant to perform a respective action.

### Claim 28

A computer-implemented method, comprising:
at an electronic device with one or more processors and memory:
displaying content on a display unit;
while displaying the content, detecting a user input; and
in response to detecting the user input:
displaying a user interface on the display unit, the user interface including a plurality of exemplary natural language requests that are contextually-related to the displayed content, wherein receiving a user utterance corresponding to one of the plurality of exemplary natural language requests causes the digital assistant to perform a respective action.
