# US 2026/0093446 A1 · Interactions with user interfaces

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0093446 A1
- Document ID: US20260093446A1
- Status: Published application
- Kind: A1
- Application number: 19/277,278
- Filed: 2025-07-22
- Published: 2026-04-02
- Earliest claimed: 2024-09-27
- Assignees: Apple Inc.
- Applicants: Apple Inc.
- Inventors: Jesse CHAND; Jonathan R. DASCOLA; Marinos BERNITSAS; Adam L. AMADIO; Danvin RUANGCHAN
- Web page: https://patentgrep.com/patent/US20260093446A1

## CPC

- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence)
- G06F 3/04845: for image manipulation, e.g. dragging, rotation, expansion or change of colour
- G06F 3/0485: Scrolling or panning
- G06F 3/165: {Management of the audio stream, e.g. setting of volume, audio stream path}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Techniques for interacting with user interfaces using user inputs are described.

## Claims

### Claim at position 1

1-85. (canceled)

### Claim 86

A computer system configured to communicate with one or more display generation components, one or more input devices, and one or more audio output devices, the computer system comprising:
one or more processors; and
memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for:
concurrently displaying, via the one or more display generation components, a first user interface object in a user interface and outputting, via the one or more audio output devices, first audio that corresponds to the first user interface object;
while concurrently displaying the first user interface object in the user interface and outputting the first audio, detecting, via the one or more input devices, a request to move the first user interface object; and
in response to detecting the request to move the first user interface object:
moving the first user interface object in accordance with the request; and
in accordance with a determination that a first set of one or more criteria is met, reducing a prominence of the first audio while continuing to display the first user interface object in the user interface.

### Claim 87 (depends on 86)

The computer system of claim 86, wherein reducing the prominence of the first audio includes reducing a volume of the first audio.

### Claim 88 (depends on 86)

The computer system of claim 86, wherein the first set of one or more criteria includes a position criterion that is based on a position of the first user interface object within a set of scrollable objects.

### Claim 89 (depends on 86)

The computer system of claim 86, wherein the first set of one or more criteria includes a size criterion that is based on a size of the first user interface object and wherein the size of the first user interface object automatically changes as the first user interface object moves.

### Claim 90 (depends on 89)

The computer system of claim 89, wherein a set of scrollable objects includes the first user interface object, the one or more programs further including instructions for:
in response to detecting the request to move the first user interface object, changing a size of the first user interface object relative to a size of one or more other objects of the set of scrollable objects.

### Claim 91 (depends on 86)

The computer system of claim 86, wherein the first set of one or more criteria includes a speed criterion that is based on a speed of movement of the first user interface object.

### Claim 92 (depends on 86)

The computer system of claim 86, wherein the first set of one or more criteria includes a set of one or more gaze criteria that is based on whether a gaze of a user is directed to the first user interface object.

### Claim 93 (depends on 86)

The computer system of claim 86, wherein the first set of one or more criteria includes a duration criterion that is met when a gaze of a user of the computer system is directed away from the first user interface object for a threshold duration of time.

### Claim 94 (depends on 86)

The computer system of claim 86, wherein the user interface is a user interface of an application, the one or more programs further including instructions for:
while outputting, via the one or more audio output devices, the first audio that corresponds to the first user interface object, detecting that a gaze of a user is not directed to interfaces of the application; and
while the gaze of the user is not directed to interfaces of the application, continuing to output, via the one or more audio output devices, the first audio that corresponds to the first user interface object.

### Claim 95 (depends on 86)

The computer system of claim 86, the one or more programs further including instructions for:
while outputting, via the one or more audio output devices, the first audio that corresponds to the first user interface object, detecting that a gaze of a user is directed to a respective user interface object that is different from the first user interface object; and
in response to detecting that the gaze of the user is directed to the respective user interface object that is different from the first user interface object and in accordance with a determination that the respective user interface object does not correspond to respective audio, continuing to output, via the one or more audio output devices, the first audio.

### Claim 96 (depends on 86)

The computer system of claim 86, the one or more programs further including instructions for:
detecting that a gaze of a user of the computer system is not directed to a set of one or more control objects that are associated with the first user interface object; and
in response to detecting that the gaze of the user of the computer system is not directed to the set of one or more control objects, reducing a prominence of the one or more control objects.

### Claim 97 (depends on 86)

The computer system of claim 86, the one or more programs further including instructions for:
detecting that a gaze of a user of the computer system is directed to a set of one or more control objects that are associated with the first user interface object; and
in response to detecting that the gaze of the user of the computer system is directed to the set of one or more control objects, increasing a prominence of the one or more control objects.

### Claim 98 (depends on 86)

The computer system of claim 86, the one or more programs further including instructions for:
while displaying the first user interface object in the user interface, detecting, via the one or more input devices, a second request to move the first user interface object; and
in response to detecting the second request to move the first user interface object:
moving the first user interface object in accordance with the second request; and
in accordance with a determination that a second set of one or more criteria is met, increasing a prominence of the first audio while continuing to display the first user interface object in the user interface.

### Claim 99 (depends on 98)

The computer system of claim 98, wherein:
the first set of one or more criteria includes a first threshold of a first type;
the second set of one or more criteria includes a second threshold of the first type; and
the first threshold is different from the second threshold.

### Claim 100 (depends on 86)

The computer system of claim 86, wherein reducing the prominence of the first audio includes gradually changing a prominence of the first audio over time.

### Claim 101 (depends on 86)

The computer system of claim 86, wherein:
the user interface includes a plurality of user interface objects, including the first user interface object and a second user interface object that is different from the first user interface object;
the first user interface object corresponds to the first audio and the second user interface object corresponds to second audio that is different from the first audio; and
the computer system outputs, via the one or more audio output devices, one primary audio from among audio corresponding to the user interface objects of the plurality of user interface objects.

### Claim 102 (depends on 101)

The computer system of claim 101, the one or more programs further including instructions for:
transitioning the primary audio from the first audio to the second audio by crossfading the first audio and the second audio, including concurrently:
reducing a volume of the first audio; and
increasing a volume of the second audio.

### Claim 103 (depends on 86)

The computer system of claim 86, wherein the first user interface object includes a first video and the first video is playing when the computer system detects the request to move the first user interface object, the one or more programs further including instructions for:
in response to detecting the request to move the first user interface object:
in accordance with a determination that a third set of one or more criteria is met, reducing a prominence of the first video while continuing to display the first user interface object in the user interface.

### Claim 104 (depends on 103)

The computer system of claim 103, wherein the first set of one or more criteria is different from the third set of one or more criteria.

### Claim 105 (depends on 103)

The computer system of claim 103, wherein:
the third set of one or more criteria includes a content-based criterion that is met when the computer system displays a respective type of content that corresponds to the user interface; and
reducing a prominence of the first video includes pausing the first video.

### Claim 106 (depends on 103)

The computer system of claim 103, wherein:
the third set of one or more criteria includes a first display criterion that is met when more than a first display threshold amount of the first user interface object moves out of a display area; and
reducing a prominence of the first video includes reducing a visual prominence of the first video.

### Claim 107 (depends on 106)

The computer system of claim 106, wherein:
reducing the prominence of the first video includes pausing the first video when more than a third display threshold amount of the first user interface object moves out of the display area;
reducing the prominence of the first audio includes pausing the first audio when more than a fourth display threshold amount of the first user interface object moves out of the display area; and
the third display threshold amount is different from the fourth display threshold amount.

### Claim 108 (depends on 106)

The computer system of claim 106, wherein reducing the prominence of the first video includes slowing down a rate of playback of the first video.

### Claim 109

A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a computer system that is in communication with one or more display generation components, one or more input devices, and one or more audio output devices, the one or more programs including instructions for:
concurrently displaying, via the one or more display generation components, a first user interface object in a user interface and outputting, via the one or more audio output devices, first audio that corresponds to the first user interface object;
while concurrently displaying the first user interface object in the user interface and outputting the first audio, detecting, via the one or more input devices, a request to move the first user interface object; and
in response to detecting the request to move the first user interface object:
moving the first user interface object in accordance with the request; and
in accordance with a determination that a first set of one or more criteria is met, reducing a prominence of the first audio while continuing to display the first user interface object in the user interface.

### Claim 110

A method, comprising:
at a computer system that is in communication with one or more display generation components, one or more input devices, and one or more audio output devices:
concurrently displaying, via the one or more display generation components, a first user interface object in a user interface and outputting, via the one or more audio output devices, first audio that corresponds to the first user interface object;
while concurrently displaying the first user interface object in the user interface and outputting the first audio, detecting, via the one or more input devices, a request to move the first user interface object; and
in response to detecting the request to move the first user interface object:
moving the first user interface object in accordance with the request; and
in accordance with a determination that a first set of one or more criteria is met, reducing a prominence of the first audio while continuing to display the first user interface object in the user interface.
