# US 2022/0035510 A1 · User interfaces for tracking and finding items

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0035510 A1
- Document ID: US20220035510A1
- Status: Published application
- Kind: A1
- Application number: 17/503,246
- Filed: 2021-10-15
- Published: 2022-02-03
- Earliest claimed: 2019-04-17
- Assignees: Apple Inc.
- Applicants: Apple Inc.
- Inventors: Arian BEHZADI; Gemma ROPER; Marcel VAN OS; Per Haakan Linus PERSSON; Pedro MARI; Stuart John WOOD; Jean-Francois M. ALBOUZE
- Web page: https://patentgrep.com/patent/US20220035510A1

## CPC

- G06F 3/04815: Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object
- G06F 3/016: {Input arrangements with force or tactile feedback as computer generated output to the user}
- G06F 3/0482: Interaction with lists of selectable items, e.g. menus
- G06F 3/0484: for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06V 20/20: in augmented reality scenes
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- G08B 5/22: using electric transmission; using electromagnetic transmission
- H04W 4/023: {using mutual or relative location information between multiple location based services [LBS] targets or of distance thresholds}
- H04W 4/029: Location-based management or tracking services
- H04W 4/80: Services using short range communication, e.g. near-field communication [NFC], radio-frequency identification [RFID] or low energy communication
- G06F 3/0346: with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
- H04W 4/02: Services making use of location information
- H04W 4/14: Short messaging services, e.g. short message services [SMS] or unstructured supplementary service data [USSD]
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G08B: SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72)

## Abstract

In some embodiments, an electronic device present user interfaces for initializing a remote locator object. In some embodiments, an electronic device presents notifications when a remote locator object is separated from the user. In some embodiments, an electronic device presents notifications when an unknown remote locator object is tracking the user. In some embodiments, an electronic device presents a user interface for a short distance locator user interface for finding a remote locator object. In some embodiments, an electronic device presents user interfaces for finding a remote locator object using a map user interface or using a short distance locator user interface.

## Claims

### Claim 1

A method, comprising:
at an electronic device with a display device and one or more input devices:
while in a remote locator object finding mode, displaying, via the display device, a first visual indicator that indicates a direction in which the remote locator object is located relative to the electronic device;
while displaying the first visual indicator, detecting a change in orientation of the electronic device; and
in response to detecting the change in orientation of the electronic device, changing an appearance of the first visual indicator to indicate a direction in which the remote locator object is located relative to the electronic device.

### Claim 2 (depends on 1)

The method of claim 1, wherein, changing the appearance of the first visual indicator to indicate the direction in which the remote locator object is located relative to the electronic device includes:
rotating the first visual indicator as the device rotates; and
in accordance with a determination that the first visual indicator has a predetermined alignment with a first portion of the device, changing a color of at least a portion of the user interface.

### Claim 3 (depends on 1)

The method of claim 1, further including generating a sequence of tactile outputs as the orientation of the device changes relative to the location associated with the remote locator object.

### Claim 4 (depends on 3)

The method of claim 3, further including in accordance with a determination that the orientation of the device relative to the location associated with the remote locator object satisfies a first orientation criteria, generating a tactile output that is different from the tactile outputs in the sequence of tactile outputs generated when the orientation of the device relative to the location associated with the remote locator object changes.

### Claim 5 (depends on 1)

The method of claim 1, further including:
prior to entering the remote locator object finding mode, displaying an information user interface for the remote locator object that includes information about the remote locator object along with a selectable user interface object for finding the remote locator object;
while displaying the information user interface for the remote locator object, detecting selection of the user interface object for finding the remote locator object; and
in response to detecting selection of the user interface object for finding the remote locator object, entering the remote locator object finding mode.

### Claim 6 (depends on 1)

The method of claim 1, wherein the first visual indicator is a simulated object and the first visual indicator changes in appearance as the orientation of the device changes so as to remain in a substantially fixed orientation relative to at least a portion of physical space.

### Claim 7 (depends on 1)

The method of claim 1, further including:
receiving a request to enter an augmented reality finding mode; and
in response to the request to enter the augmented reality finding mode, displaying a representation of a field of view of one or more cameras of the device and displaying a virtual object at a location that is determined based on the location associated with the remote locator object.

### Claim 8 (depends on 7)

The method of claim 7, wherein the request to enter the augmented reality finding mode includes changing an orientation of the device from a first orientation to a second orientation

### Claim 9 (depends on 7)

The method of claim 7, further including, in response to the request to enter the augmented reality finding mode, displaying an animation of the first visual indicator flying toward the location associated with the remote locator object in the physical space.

### Claim 10 (depends on 7)

The method of claim 7, further including generating a tactile output corresponding to entering the augmented reality mode.

### Claim 11 (depends on 1)

The method of claim 1, wherein the remote locator object finder mode includes a selectable affordance for requesting the generation of a sound at the remote locator object.

### Claim 12 (depends on 1)

The method of claim 1, wherein the remote locator object finding mode includes a text description of directions from the location of the device to the location associated with the remote locator object.

### Claim 13 (depends on 1)

The method of claim 1, wherein the remote locator object finding mode includes a stationary element at a predefined portion of the user interface and an element that changes in size as a predefined portion of the first visual indicator moves closer to and farther away from the stationary element.

### Claim 14 (depends on 13)

The method of claim 13, further including when the device is oriented such that the predefined portion of the first visual indicator has a predetermined orientation relative to the stationary element:
changing an appearance of the stationary element; and
generating a tactile output corresponding to the change in appearance of the stationary element.

### Claim 15 (depends on 14)

The method of claim 14, wherein the visual indicator has the predetermined orientation when it is within a predetermined range of orientations.

### Claim 16 (depends on 14)

The method of claim 14, further including generating a sequence of discrete tactile outputs while the device is oriented such that the predefined portion of the first visual indicator has a predetermined orientation relative to the stationary element, wherein a spacing between the discrete tactile outputs changes as the distance between the device and the remote locator object changes.

### Claim 17 (depends on 1)

The method of claim 1, further including, when a distance between the device and the remote locator object decreases below a predetermined distance threshold, replacing display of the first visual indicator that changes in appearance to indicate the direction in which the remote locator object is located relative to the electronic device with a second visual indicator that changes in appearance based on a change in distance between the device and the remote locator object without regard to a direction in which the remote locator object is located relative to the electronic device.

### Claim 18 (depends on 1)

The method of claim 1, further including:
in accordance with a determination that a distance between the device and the remote locator object is above a predetermined threshold, generating a sequence of discrete tactile outputs where a spacing between the discrete tactile outputs changes based on a change in distance between the device and the remote locator object; and
in accordance with a determination that the distance between the device and the remote locator object is below the predetermined threshold, ceasing generating the sequence of discrete tactile outputs and generating a continuous tactile output.

### Claim 19 (depends on 18)

The method of claim 18, wherein the continuous tactile output changes in tactile output property based on changes in distance between the device and the remote locator object.

### Claim 20 (depends on 1)

The method of claim 1, further comprising:
while in the remote locator object finding mode in which the first visual indicator that indicates the direction in which the remote locator object is located relative to the electronic device is displayed via the display device, detecting that the remote locator object is at a location in an environment of the electronic device that corresponds to a field of view of a camera of the electronic device; and
in response to detecting that the remote locator object is at the location in the environment of the electronic device that corresponds to the field of view of the camera of the electronic device, displaying, via the display device, a camera user interface that displays content captured by the camera of the electronic device, and a first visual indication, overlaid on the content captured by the camera, at a location in the content that corresponds to the location of the remote locator object in the field of view of the camera.

### Claim 21 (depends on 20)

The method of claim 20, further comprising:
in response to a distance between the electronic device and the remote locator object decreasing below a predetermined distance threshold, replacing display of the first visual indicator with a second visual indicator that surrounds the location in the content that corresponds to the location of the remote locator object in the field of view of the camera.

### Claim 22

An electronic device, comprising:
one or more processors;
memory; and
one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for:
while in a remote locator object finding mode, displaying, via a display device, a first visual indicator that indicates a direction in which the remote locator object is located relative to the electronic device;
while displaying the first visual indicator, detecting a change in orientation of the electronic device; and
in response to detecting the change in orientation of the electronic device, changing an appearance of the first visual indicator to indicate a direction in which the remote locator object is located relative to the electronic device.

### Claim 23

A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors of an electronic device, cause the electronic device to perform a method comprising:
while in a remote locator object finding mode, displaying, via a display device, a first visual indicator that indicates a direction in which the remote locator object is located relative to the electronic device;
while displaying the first visual indicator, detecting a change in orientation of the electronic device; and
in response to detecting the change in orientation of the electronic device, changing an appearance of the first visual indicator to indicate a direction in which the remote locator object is located relative to the electronic device.
