# US 2026/0093336 A1 · Caching and referencing strategies for interaction with informational content in a physical environment

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0093336 A1
- Document ID: US20260093336A1
- Status: Published application
- Kind: A1
- Application number: 19/330,570
- Filed: 2025-09-16
- Published: 2026-04-02
- Earliest claimed: 2024-09-28
- Assignees: Apple Inc.
- Applicants: Apple Inc.
- Inventors: Ashwin K. VIJAY
- Web page: https://patentgrep.com/patent/US20260093336A1

## CPC

- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06F 16/24552: {Database cache management}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Some examples of the disclosure are directed to systems and methods for capturing and caching one or more first optical captures of an object in a physical environment, and subsequently capturing one or more second optical captures after one or more portions of a user are detected to be directed to the first object. When the one or more portions of the user are determined to satisfy certain criteria (e.g., occluding a first region of the first object), the electronic device performs one or more operations on the one or more first optical captures including recognizing, generating representations of, displaying related information, and/or saving informational content associated with the first object, including informational content occluded by the one or more portions of a user.

## Claims

### Claim 1

A method comprising:
at a first electronic device in communication with one or more input devices including one or more optical sensors and a memory:
capturing one or more first optical captures of one or more first objects in a first physical environment;
predicting one or more interactions with the one or more first objects in the first physical environment, wherein at least a first interaction of the one or more interactions corresponds to a request for first informational content corresponding to at least a first object of the one or more first objects;
after predicting the one or more interactions with the one or more first objects in the first physical environment and prior to receiving an input corresponding to the first interaction with the first object:
obtaining, at a first time, the first informational content corresponding to the first interaction and to the first object; and
storing, in the memory, the first informational content corresponding to the first interaction and to the first object;
after storing the first informational content, receiving the input corresponding to the first interaction with the first object; and
in response to receiving the input corresponding to the first interaction with the first object, and in accordance with a determination that one or more first criteria are satisfied:
obtaining, at a second time after the first time, the first informational content corresponding to the first interaction with the first object from the memory; and
presenting the first informational content corresponding to the first interaction with the first object.

### Claim 2 (depends on 1)

The method of claim 1, wherein obtaining, at the first time, the first informational content corresponding to the first interaction and to the first object includes accessing the informational content corresponding to at least the first object of the one or more first objects or initiating presentation of the informational content corresponding to at least the first object of the one or more first objects.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
after storing the first informational content, capturing one or more second optical captures of the one or more first objects in the first physical environment,
wherein the input corresponding to the first interaction with the first object includes an object-interaction gesture detected in at least one of the one or more second optical captures.

### Claim 4 (depends on 3)

The method of claim 3, wherein the one or more first criteria include a criterion that is satisfied when attention of a user of the first electronic device is directed to the first object.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
receiving an input corresponding to a second interaction with a second object, different from the one or more first objects, wherein the second interaction corresponds to a request for second informational content; and
in response to receiving the input corresponding to the second interaction with the second object, and in accordance with a determination that one or more second criteria are satisfied:
initiating a request for the second informational content corresponding to the second interaction with the second object from a second electronic device, different from the first electronic device.

### Claim 6 (depends on 1)

The method of claim 1, wherein predicting the one or more interactions with the one or more first objects in the first physical environment includes predicting a second interaction, different from the first interaction, with the first object corresponding to a request for second informational content corresponding to the first object, and the method further comprising:
after predicting the one or more interactions with the one or more first objects and prior to receiving an input corresponding to the second interaction with the first object:
obtaining, at a third time, the second informational content corresponding to the second interaction and to the first object; and
storing, in the memory, the second informational content corresponding to the second interaction and to the first object;
after storing the second informational content, receiving the input corresponding to the second interaction with the first object; and
in response to receiving the input corresponding to the second interaction with the first object, and in accordance with a determination that the one or more first criteria are satisfied:
obtaining, at a fourth time, the second informational content corresponding to the second interaction with the first object from the memory; and
presenting the second informational content corresponding to the second interaction with the first object.

### Claim 7 (depends on 1)

The method of claim 1, wherein predicting the one or more interactions with the one or more first objects in the first physical environment includes predicting a second interaction with a second object of the one or more first objects, different from the first object, corresponding to a request for second informational content corresponding to the second object, and the method further comprising:
after predicting the one or more interactions with the one or more first objects and prior to receiving an input corresponding to the second interaction with the second object:
obtaining, at a third time, the second informational content corresponding to the second interaction and to the second object; and
storing, in the memory, the second informational content corresponding to the second interaction and to the second object;
after storing the second informational content, receiving the input corresponding to the second interaction with the second object; and
in response to receiving the input corresponding to the second interaction with the second object, and in accordance with a determination that the one or more first criteria are satisfied:
obtaining, at a fourth time, the second informational content corresponding to the second interaction with the second object from the memory; and
presenting the second informational content corresponding to the second interaction with the second object.

### Claim 8 (depends on 1)

The method of claim 1, wherein predicting one or more interactions with the one or more first objects in the first physical environment includes obtaining a semantic heatmap of the one or more interactions corresponding to the one or more first objects in the first physical environment.

### Claim 9

A first electronic device, comprising:
one or more processors;
memory; and
one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for:
capturing one or more first optical captures of one or more first objects in a first physical environment;
predicting one or more interactions with the one or more first objects in the first physical environment, wherein at least a first interaction of the one or more interactions corresponds to a request for first informational content corresponding to at least a first object of the one or more first objects;
after predicting the one or more interactions with the one or more first objects in the first physical environment and prior to receiving an input corresponding to the first interaction with the first object:
obtaining, at a first time, the first informational content corresponding to the first interaction and to the first object; and
storing, in the memory, the first informational content corresponding to the first interaction and to the first object;
after storing the first informational content, receiving the input corresponding to the first interaction with the first object; and
in response to receiving the input corresponding to the first interaction with the first object, and in accordance with a determination that one or more first criteria are satisfied:
obtaining, at a second time after the first time, the first informational content corresponding to the first interaction with the first object from the memory; and
presenting the first informational content corresponding to the first interaction with the first object.

### Claim 10 (depends on 9)

The first electronic device of claim 9, wherein obtaining, at the first time, the first informational content corresponding to the first interaction and to the first object includes accessing the informational content corresponding to at least the first object of the one or more first objects or initiating presentation of the informational content corresponding to at least the first object of the one or more first objects.

### Claim 11 (depends on 9)

The first electronic device of claim 9, wherein the one or more programs further include instructions for:
after storing the first informational content, capturing one or more second optical captures of the one or more first objects in the first physical environment,
wherein the input corresponding to the first interaction with the first object includes an object-interaction gesture detected in at least one of the one or more second optical captures.

### Claim 12 (depends on 11)

The first electronic device of claim 11, wherein the one or more first criteria include a criterion that is satisfied when attention of a user of the first electronic device is directed to the first object.

### Claim 13 (depends on 9)

The first electronic device of claim 9, wherein the one or more programs further include instructions for:
receiving an input corresponding to a second interaction with a second object, different from the one or more first objects, wherein the second interaction corresponds to a request for second informational content; and
in response to receiving the input corresponding to the second interaction with the second object, and in accordance with a determination that one or more second criteria are satisfied:
initiating a request for the second informational content corresponding to the second interaction with the second object from a second electronic device, different from the first electronic device.

### Claim 14 (depends on 9)

The first electronic device of claim 9, wherein predicting the one or more interactions with the one or more first objects in the first physical environment includes predicting a second interaction, different from the first interaction, with the first object corresponding to a request for second informational content corresponding to the first object, and the one or more programs further including instructions for:
after predicting the one or more interactions with the one or more first objects and prior to receiving an input corresponding to the second interaction with the first object:
obtaining, at a third time, the second informational content corresponding to the second interaction and to the first object; and
storing, in the memory, the second informational content corresponding to the second interaction and to the first object;
after storing the second informational content, receiving the input corresponding to the second interaction with the first object; and
in response to receiving the input corresponding to the second interaction with the first object, and in accordance with a determination that the one or more first criteria are satisfied:
obtaining, at a fourth time, the second informational content corresponding to the second interaction with the first object from the memory; and
presenting the second informational content corresponding to the second interaction with the first object.

### Claim 15 (depends on 9)

The first electronic device of claim 9, wherein predicting the one or more interactions with the one or more first objects in the first physical environment includes predicting a second interaction with a second object of the one or more first objects, different from the first object, corresponding to a request for second informational content corresponding to the second object, and the one or more programs further including instructions for:
after predicting the one or more interactions with the one or more first objects and prior to receiving an input corresponding to the second interaction with the second object:
obtaining, at a third time, the second informational content corresponding to the second interaction and to the second object; and
storing, in the memory, the second informational content corresponding to the second interaction and to the second object;
after storing the second informational content, receiving the input corresponding to the second interaction with the second object; and
in response to receiving the input corresponding to the second interaction with the second object, and in accordance with a determination that the one or more first criteria are satisfied:
obtaining, at a fourth time, the second informational content corresponding to the second interaction with the second object from the memory; and
presenting the second informational content corresponding to the second interaction with the second object.

### Claim 16 (depends on 9)

The first electronic device of claim 9, wherein predicting one or more interactions with the one or more first objects in the first physical environment includes obtaining a semantic heatmap of the one or more interactions corresponding to the one or more first objects in the first physical environment.

### Claim 17

A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors of a first electronic device, cause the first electronic device to:
capture one or more first optical captures of one or more first objects in a first physical environment;
predict one or more interactions with the one or more first objects in the first physical environment, wherein at least a first interaction of the one or more interactions corresponds to a request for first informational content corresponding to at least a first object of the one or more first objects;
after predicting the one or more interactions with the one or more first objects in the first physical environment and prior to receiving an input corresponding to the first interaction with the first object:
obtain, at a first time, the first informational content corresponding to the first interaction and to the first object; and
store, in memory, the first informational content corresponding to the first interaction and to the first object;
after storing the first informational content, receiving the input corresponding to the first interaction with the first object; and
in response to receiving the input corresponding to the first interaction with the first object, and in accordance with a determination that one or more first criteria are satisfied:
obtain, at a second time after the first time, the first informational content corresponding to the first interaction with the first object from the memory; and
present the first informational content corresponding to the first interaction with the first object.

### Claim 18 (depends on 17)

The non-transitory computer readable storage medium of claim 17, wherein obtaining, at the first time, the first informational content corresponding to the first interaction and to the first object includes accessing the informational content corresponding to at least the first object of the one or more first objects or initiating presentation of the informational content corresponding to at least the first object of the one or more first objects.

### Claim 19 (depends on 17)

The non-transitory computer readable storage medium of claim 17, wherein the instructions further cause the first electronic device to:
after storing the first informational content, capture one or more second optical captures of the one or more first objects in the first physical environment,
wherein the input corresponding to the first interaction with the first object includes an object-interaction gesture detected in at least one of the one or more second optical captures.

### Claim 20 (depends on 19)

The non-transitory computer readable storage medium of claim 19, wherein the one or more first criteria include a criterion that is satisfied when attention of a user of the first electronic device is directed to the first object.

### Claim 21 (depends on 17)

The non-transitory computer readable storage medium of claim 17, wherein the instructions further cause the first electronic device to:
receive an input corresponding to a second interaction with a second object, different from the one or more first objects, wherein the second interaction corresponds to a request for second informational content; and
in response to receiving the input corresponding to the second interaction with the second object, and in accordance with a determination that one or more second criteria are satisfied:
initiate a request for the second informational content corresponding to the second interaction with the second object from a second electronic device, different from the first electronic device.

### Claim 22 (depends on 17)

The non-transitory computer readable storage medium of claim 17, wherein predicting the one or more interactions with the one or more first objects in the first physical environment includes predicting a second interaction, different from the first interaction, with the first object corresponding to a request for second informational content corresponding to the first object, and the instructions further cause the first electronic device to:
after predicting the one or more interactions with the one or more first objects and prior to receiving an input corresponding to the second interaction with the first object:
obtain, at a third time, the second informational content corresponding to the second interaction and to the first object; and
store, in the memory, the second informational content corresponding to the second interaction and to the first object;
after storing the second informational content, receiving the input corresponding to the second interaction with the first object; and
in response to receiving the input corresponding to the second interaction with the first object, and in accordance with a determination that the one or more first criteria are satisfied:
obtain, at a fourth time, the second informational content corresponding to the second interaction with the first object from the memory; and
present the second informational content corresponding to the second interaction with the first object.

### Claim 23 (depends on 17)

The non-transitory computer readable storage medium of claim 17, wherein predicting the one or more interactions with the one or more first objects in the first physical environment includes predicting a second interaction with a second object of the one or more first objects, different from the first object, corresponding to a request for second informational content corresponding to the second object, and the instructions further cause the first electronic device to:
after predicting the one or more interactions with the one or more first objects and prior to receiving an input corresponding to the second interaction with the second object:
obtain, at a third time, the second informational content corresponding to the second interaction and to the second object; and
store, in the memory, the second informational content corresponding to the second interaction and to the second object;
after storing the second informational content, receiving the input corresponding to the second interaction with the second object; and
in response to receiving the input corresponding to the second interaction with the second object, and in accordance with a determination that the one or more first criteria are satisfied:
obtaining, at a fourth time, the second informational content corresponding to the second interaction with the second object from the memory; and
presenting the second informational content corresponding to the second interaction with the second object.

### Claim 24 (depends on 17)

The non-transitory computer readable storage medium of claim 17, wherein predicting one or more interactions with the one or more first objects in the first physical environment includes obtaining a semantic heatmap of the one or more interactions corresponding to the one or more first objects in the first physical environment.
