# US 2024/0004532 A1 · Interactions between an input device and an electronic device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0004532 A1
- Document ID: US20240004532A1
- Status: Published application
- Kind: A1
- Application number: 18/315,251
- Filed: 2023-05-10
- Published: 2024-01-04
- Earliest claimed: 2022-05-10
- Assignees: Apple Inc.
- Applicants: Apple Inc.
- Inventors: Christopher D. SOLI; Marcos ALONSO; Daniel T. PRESTON; Jennifer P. CHEN; Peder BLEKKEN
- Web page: https://patentgrep.com/patent/US20240004532A1

## CPC

- G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence)
- G06F 3/0488: using a touch-screen or digitiser, e.g. input of commands through traced gestures
- G06F 3/0416: {Control or interface arrangements specially adapted for digitisers}
- G06F 3/0441: {using active external devices, e.g. active pens, for receiving changes in electrical potential transmitted by the digitiser, e.g. tablet driving signals}
- G06F 3/0442: {using active external devices, e.g. active pens, for transmitting changes in electrical potential to be received by the digitiser}
- G06F 3/04812: Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects
- G06F 3/04883: for inputting data by handwriting, e.g. gesture or text
- H04L 12/2809: {indicating that an appliance service is present in a home automation network (monitoring functionality H04L43/0817; discovery or management thereof, e.g. service location protocol [SLP] or web services, H04L67/51)}
- H04L 12/281: {indicating a format for calling an appliance service function in a home automation network (for remote control or remote monitoring of applications H04L67/025)}
- H04L 67/10: in which an application is distributed across nodes in the network (software deployment G06F8/60; multiprogramming arrangements G06F9/46)
- H04W 4/70: Services for machine-to-machine communication [M2M] or machine type communication [MTC]
- H04W 12/50: Secure pairing of devices
- G06F 2203/04101: 2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
- G06F 2203/04106: Multi-sensing digitiser, i.e. digitiser using at least two different sensing technologies simultaneously or alternatively, e.g. for detecting pen and finger, for saving power or for improving position detection
- G06F 2203/04108: Touchless 2D- digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface without distance measurement in the Z direction
- H04W 4/33: for indoor environments, e.g. buildings
- H04W 4/38: for collecting sensor information
- H04W 12/009: {specially adapted for networks, e.g. wireless sensor networks, ad-hoc networks, RFID networks or cloud networks}
- H04W 84/12: WLAN [Wireless Local Area Networks]
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72)

## Abstract

Some embodiments described in this disclosure are directed to displaying additional controls and/or information when an input device such as a stylus is hovering over a user interface displayed by an electronic device. Some embodiments described in this disclosure are directed to providing feedback about the pose of an input device relative to a surface. Some embodiments of the disclosure are directed to performing contextual actions in response to input provided from an input device. Some embodiments of the disclosure are directed to providing handwritten input for conversion into font-based text using an input device.

## Claims

### Claim 1

A method comprising:
at an electronic device in communication with a display generation component and one or more sensors:
displaying, via the display generation component, a user interface including a first user interface object;
while displaying, via the display generation component, the user interface including the first user interface object, detecting, via the one or more sensors, a respective object in proximity to, but not in contact with, a surface associated with the user interface; and
in response to detecting the respective object in proximity to, but not in contact with, the surface:
in accordance with a determination that the respective object in proximity to the surface is an input device in communication with the electronic device, and that a position of the input device corresponds to the first user interface object, displaying, in the user interface, a first selectable option that is selectable to perform a first operation associated with the first user interface object; and
in accordance with a determination that the respective object in proximity to the surface is not an input device in communication with the electronic device, forgoing the displaying, in the user interface, of the first selectable option that is selectable to perform the first operation associated with the first user interface object.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
in response to detecting the respective object in proximity to, but not in contact with, the surface, in accordance with the determination that the respective object in proximity to the surface is the input device in communication with the electronic device, and that the position of the input device corresponds to the first user interface object, modifying a visual characteristic of the first user interface object to indicate that the first user interface object is selectable.

### Claim 3 (depends on 1)

The method of claim 1, wherein the first user interface object is associated with a selection of a first region of the user interface and not a second region of the user interface, different from the first region of the user interface, and the first user interface object is interactable to modify a region of the user interface that is selected by the first user interface object, the method further comprising:
in response to detecting the respective object in proximity to, but not in contact with, the surface, in accordance with the determination that the respective object in proximity to the surface is the input device in communication with the electronic device, and that the position of the input device corresponds to the first user interface object, displaying, via the display generation component, a visual indication associated with one or more directions of the modification of the region of the user interface that is selected by the first user interface object.

### Claim 4 (depends on 1)

The method of claim 1, wherein the first user interface object corresponds to a first portion of the user interface, and the first selectable option is associated with ceasing display of the first portion of the user interface, the method further comprising:
while displaying the first selectable option in accordance with the determination that the respective object in proximity to the surface is the input device in communication with the electronic device, and that the position of the input device corresponds to the first user interface object, receiving, via the one or more sensors, one or more inputs corresponding to a selection of the first selectable option; and
in response to receiving the one or more inputs corresponding to the selection of the first selectable option, ceasing display of the first portion of the user interface.

### Claim 5 (depends on 1)

The method of claim 1, wherein the first user interface object includes a content entry region that includes content, and the first selectable option is associated with ceasing display of the content in the content entry region, the method further comprising:
while displaying the first selectable option in accordance with the determination that the respective object in proximity to the surface is the input device in communication with the electronic device, and that the position of the input device corresponds to the first user interface object, receiving, via the one or more sensors, one or more inputs corresponding to a selection of the first selectable option; and
in response to receiving the one or more inputs corresponding to the selection of the first selectable option, ceasing display of the content within the content entry region.

### Claim 6 (depends on 1)

The method of claim 1, wherein the first user interface object is associated with presenting media content, and the first selectable option is associated with modifying playback of the media content, the method further comprising:
while displaying the first selectable option in accordance with the determination that the respective object in proximity to the surface is the input device in communication with the electronic device, and that the position of the input device corresponds to the first user interface object, receiving, via the one or more sensors, one or more inputs corresponding to a selection of the first selectable option; and
in response to receiving the one or more inputs corresponding to the selection of the first selectable option, modifying playback of the media content.

### Claim 7 (depends on 1)

The method of claim 1, further comprising:
in response to detecting the respective object in proximity to, but not in contact with, the surface, in accordance with the determination that the respective object in proximity to the surface is the input device in communication with the electronic device, and that the position of the input device corresponds to the first user interface object:
in accordance with a determination that the position of the input device satisfies one or more criteria, including a criterion that is satisfied when the position of the input device corresponds to the first user interface object for longer than a threshold amount of time, displaying, via the display generation component, information associated with the first user interface object.

### Claim 8 (depends on 1)

The method of claim 1, wherein the first user interface object includes a content entry region, the method further comprising:
in response to detecting the respective object in proximity to, but not in contact with, the surface, in accordance with the determination that the respective object in proximity to the surface is the input device in communication with the electronic device, and that the position of the input device corresponds to the first user interface object, displaying, via the display generation component, a visual indication of a content insertion cursor in the content entry region at a location corresponding to the input device;
while displaying the visual indication, receiving, via the one or more sensors, one or more inputs corresponding to a selection of the visual indication; and
in response to receiving the one or more inputs corresponding to the selection of the visual indication, displaying, via the display generation component, the content insertion cursor at the location corresponding to the input device in the content entry region.

### Claim 9 (depends on 1)

The method of claim 1, wherein the first user interface object includes content, the method further comprising:
in response to detecting the respective object in proximity to, but not in contact with, the surface, in accordance with the determination that the respective object in proximity to the surface is the input device in communication with the electronic device, and that the position of the input device corresponds to the content:
in accordance with a determination that the content is non-editable content, displaying, via the display generation component, a visual indication of a content selection cursor in the content at a location corresponding to the input device.

### Claim 10 (depends on 9)

The method of claim 9, further comprising:
in response to detecting the respective object in proximity to, but not in contact with, the surface, in accordance with the determination that the respective object in proximity to the surface is the input device in communication with the electronic device, and that the position of the input device corresponds to the content:
in accordance with a determination that the content is editable content, forgoing displaying, via the display generation component, the visual indication of the content selection cursor in the content at the location corresponding to the input device.

### Claim 11 (depends on 1)

The method of claim 1, wherein before detecting the respective object in proximity to, but not in contact with, the surface, the first user interface object is displayed with a first amount of separation from a backplane, the method further comprising:
in response to detecting the respective object in proximity to, but not in contact with, the surface, in accordance with the determination that the respective object in proximity to the surface is the input device in communication with the electronic device, and that the position of the input device corresponds to the first user interface object, displaying the first user interface object with a second amount of separation, greater than the first amount of separation, from the backplane.

### Claim 12 (depends on 1)

The method of claim 1, further comprising:
While displaying, via the display generation component, the user interface including the first user interface object, wherein the first user interface object has a first visual appearance in which a first visual characteristic has a first value, detecting, via a cursor control input device, a first input corresponding to movement of a cursor from a location away from the first user interface object to the first user interface object; and
in response to detecting the first input, moving the cursor to the first user interface object, and displaying the first user interface object with a second visual appearance in which the first visual characteristic has a second value, different from the first value,
wherein while the position of the input device corresponds to the first user interface object, the first user interface object is displayed with a third visual appearance in which the first visual characteristic has the second value.

### Claim 13 (depends on 12)

The method of claim 12, further comprising:
in response to detecting the first input, displaying the first user interface object with a parallax effect based on movement of the cursor while the cursor is located at the first user interface object, wherein displaying the first user interface object with the third visual appearance in accordance with the determination that the position of the input device corresponds to the first user interface object does not include displaying the first user interface object with the parallax effect based on movement of the input device while the position of the input device corresponds to the first user interface object.

### Claim 14 (depends on 12)

The method of claim 12, further comprising:
In response to detecting the first input, displaying the first user interface object with a lighting effect based on movement of the cursor while the cursor is located at the first user interface object, wherein displaying the first user interface object with the third visual appearance in accordance with the determination that the position of the input device corresponds to the first user interface object does not include displaying the first user interface object with the lighting effect based on movement of the input device while the position of the input device corresponds to the first user interface object.

### Claim 15 (depends on 1)

The method of claim 1, wherein the first user interface object corresponds to a link to content, the method further comprising:
in response to detecting the respective object in proximity to, but not in contact with, the surface, in accordance with the determination that the respective object in proximity to the surface is the input device in communication with the electronic device, and that the position of the input device corresponds to the first user interface object, modifying a visual appearance of the first user interface object.

### Claim 16

An electronic device, comprising:
one or more processors;
memory; and
one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for:
displaying, via a display generation component, a user interface including a first user interface object;
while displaying, via the display generation component, the user interface including the first user interface object, detecting, via one or more sensors, a respective object in proximity to, but not in contact with, a surface associated with the user interface; and
in response to detecting the respective object in proximity to, but not in contact with, the surface:
in accordance with a determination that the respective object in proximity to the surface is an input device in communication with the electronic device, and that a position of the input device corresponds to the first user interface object, displaying, in the user interface, a first selectable option that is selectable to perform a first operation associated with the first user interface object; and
in accordance with a determination that the respective object in proximity to the surface is not an input device in communication with the electronic device, forgoing the displaying, in the user interface, of the first selectable option that is selectable to perform the first operation associated with the first user interface object.

### Claim 17

A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors of an electronic device, cause the electronic device to perform a method comprising:
displaying, via a display generation component, a user interface including a first user interface object;
while displaying, via the display generation component, the user interface including the first user interface object, detecting, via one or more sensors, a respective object in proximity to, but not in contact with, a surface associated with the user interface; and in response to detecting the respective object in proximity to, but not in contact with, the surface:
in accordance with a determination that the respective object in proximity to the surface is an input device in communication with the electronic device, and that a position of the input device corresponds to the first user interface object, displaying, in the user interface, a first selectable option that is selectable to perform a first operation associated with the first user interface object; and
in accordance with a determination that the respective object in proximity to the surface is not an input device in communication with the electronic device, forgoing the displaying, in the user interface, of the first selectable option that is selectable to perform the first operation associated with the first user interface object.

### Claim at position 18

-87. (canceled)
