# US 2026/0093326 A1 · Tracking of physical and virtual objects of attention with associated detection of trigger mechanism activation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0093326 A1
- Document ID: US20260093326A1
- Status: Published application
- Kind: A1
- Application number: 18/898,875
- Filed: 2024-09-27
- Published: 2026-04-02
- Earliest claimed: 2024-09-27
- Assignees: Dell Products L.P.
- Applicants: Dell Products L.P.
- Inventors: Zijia Wang; Pedro Fernandez Orellana; Ahmed Khalid
- Web page: https://patentgrep.com/patent/US20260093326A1

## CPC

- G06F 3/015: {Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

An apparatus in one embodiment comprises at least one processing device that includes a processor coupled to memory. The at least one processing device is configured to identify a plurality of objects of attention utilizing multiple sensors of a user device, the plurality of objects of attention comprising at least one physical object in an environment outside of the user device and at least one virtual object presented on a display screen of the user device. The at least one processing device is further configured to populate a data structure with entries characterizing respective ones of the plurality of objects of attention, to detect activation of at least one trigger mechanism associated with the user device, and to generate a response to the activated trigger mechanism based at least in part on at least one of the identified objects of attention characterized by a corresponding entry in the data structure.

## Claims

### Claim 1

An apparatus comprising:
at least one processing device comprising a processor coupled to a memory;
the at least one processing device being configured:
to identify a plurality of objects of attention utilizing multiple sensors of a user device, the plurality of objects of attention comprising at least one physical object in an environment outside of the user device and at least one virtual object presented on a display screen of the user device;
to populate a data structure with entries characterizing respective ones of the plurality of objects of attention;
to detect activation of at least one trigger mechanism associated with the user device; and
to generate a response to the activated trigger mechanism based at least in part on at least one of the identified objects of attention characterized by a corresponding entry in the data structure.

### Claim 2 (depends on 1)

The apparatus of claim 1 wherein the at least one processing device comprises at least one of the user device and a cloud-based processing device configured to communicate with the user device over a network.

### Claim 3 (depends on 1)

The apparatus of claim 1 wherein entries of the data structure characterize respective snapshots of user attention at respective points in time.

### Claim 4 (depends on 1)

The apparatus of claim 1 wherein the data structure comprises an attention log that includes a plurality of entries for respective ones of the identified objects of interest arranged in temporal order.

### Claim 5 (depends on 4)

The apparatus of claim 4 wherein the attention log comprises a first-in first-out (FIFO) buffer of entries for a sliding time window.

### Claim 6 (depends on 4)

The apparatus of claim 4 wherein a given one of the entries of the attention log comprises at least a subset of one or more spatial coordinates of the identified object of attention, a timestamp associated with identification of the object of attention, bounding box information characterizing a region occupied by the identified object of attention, and an addressable description of the identified object of attention.

### Claim 7 (depends on 1)

The apparatus of claim 1 wherein the at least one trigger mechanism comprises at least one proactive trigger mechanism and at least one reactive trigger mechanism.

### Claim 8 (depends on 7)

The apparatus of claim 7 wherein the at least one proactive trigger mechanism comprises a trigger mechanism based at least in part on a wearable sensor that is part of the user device or part of another associated device in communication with the user device.

### Claim 9 (depends on 8)

The apparatus of claim 8 wherein the wearable sensor comprises at least an electroencephalogram (EEG) sensor.

### Claim 10 (depends on 7)

The apparatus of claim 7 wherein the at least one reactive trigger mechanism comprises a trigger mechanism based at least in part on a voice sensor that is part of the user device or part of another associated device in communication with the user device.

### Claim 11 (depends on 10)

The apparatus of claim 10 wherein the at least one processing device is configured to interpret one or more voice commands at least in part by converting spoken input of a user as detected by the voice sensor into text, parsing the text using one or more natural language processing (NLP) techniques to extract intent relating to a corresponding voice command and any associated object references, and matching the extracted intent to one or more entries of the data structure.

### Claim 12 (depends on 1)

The apparatus of claim 1 wherein the at least one processing device is further configured to perform a certainty assessment by processing one or more outputs generated based at least in part on the one or more trigger mechanisms against one or more respective corresponding confidence thresholds and wherein the response is generated based at least in part on results of the certainty assessment.

### Claim 13 (depends on 1)

The apparatus of claim 1 wherein the at least one processing device is further configured to cross-reference one or more outputs generated based at least in part on the one or more trigger mechanisms against one or more entries of the data structure.

### Claim 14 (depends on 1)

The apparatus of claim 1 wherein the at least one processing device is further configured, responsive to detection of an ambiguity between an output generated based at least in part on a first one of the one or more trigger mechanisms and an output generated based at least in part on a second one of the one or more trigger mechanisms, to request additional input from a user and to feed back at least portions of the additional input to one or more machine learning algorithms associated with the one or more trigger mechanisms.

### Claim 15

A computer program product comprising a non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code when executed by at least one processing device causes the at least one processing device:
to identify a plurality of objects of attention utilizing multiple sensors of a user device, the plurality of objects of attention comprising at least one physical object in an environment outside of the user device and at least one virtual object presented on a display screen of the user device;
to populate a data structure with entries characterizing respective ones of the plurality of objects of attention;
to detect activation of at least one trigger mechanism associated with the user device; and
to generate a response to the activated trigger mechanism based at least in part on at least one of the identified objects of attention characterized by a corresponding entry in the data structure.

### Claim 16 (depends on 15)

The computer program product of claim 15 wherein the data structure comprises an attention log that includes a plurality of entries for respective ones of the identified objects of interest arranged in temporal order.

### Claim 17 (depends on 15)

The computer program product of claim 15 wherein the at least one trigger mechanism comprises at least one proactive trigger mechanism and at least one reactive trigger mechanism.

### Claim 18

A method comprising:
identifying a plurality of objects of attention utilizing multiple sensors of a user device, the plurality of objects of attention comprising at least one physical object in an environment outside of the user device and at least one virtual object presented on a display screen of the user device;
populating a data structure with entries characterizing respective ones of the plurality of objects of attention;
detecting activation of at least one trigger mechanism associated with the user device; and
generating a response to the activated trigger mechanism based at least in part on at least one of the identified objects of attention characterized by a corresponding entry in the data structure;
wherein the method is performed by at least one processing device comprising a processor coupled to a memory.

### Claim 19 (depends on 18)

The method of claim 18 wherein the data structure comprises an attention log that includes a plurality of entries for respective ones of the identified objects of interest arranged in temporal order.

### Claim 20 (depends on 18)

The method of claim 18 wherein the at least one trigger mechanism includes at least one proactive trigger mechanism and at least one reactive trigger mechanism.
