# US 2026/0073642 A1 · Physical and virtual object attention tracking for a user device comprising multiple sensors

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0073642 A1
- Document ID: US20260073642A1
- Status: Published application
- Kind: A1
- Application number: 18/829,568
- Filed: 2024-09-10
- Published: 2026-03-12
- Earliest claimed: 2024-09-10
- Assignees: Dell Products L.P.
- Applicants: Dell Products L.P.
- Inventors: Pedro Fernandez Orellana; Zijia Wang; Ahmed Khalid
- Web page: https://patentgrep.com/patent/US20260073642A1

## CPC

- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06F 3/14: Digital output to display device {; Cooperation and interconnection of the display device with other functional units}
- G06V 20/20: in augmented reality scenes
- G06V 2201/07: Target detection
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

An apparatus comprises at least one processing device that includes a processor coupled to a memory. The at least one processing device is configured to obtain first sensor information from at least one user-facing sensor of a user device, to obtain second sensor information from at least one environment-facing sensor of the user device, and to process the first sensor information and the second sensor information to identify an object of user attention, the object comprising one of a physical object in an environment outside of the user device and a virtual object presented on a display screen of the user device. In some embodiments, the user device comprises a laptop computer, with the user-facing sensor being arranged on a display screen side of a cover of the laptop computer and the environment-facing sensor being arranged on an opposite side of the cover relative to the display screen side.

## Claims

### Claim 1

An apparatus comprising:
at least one processing device comprising a processor coupled to a memory;
the at least one processing device being configured:
to obtain first sensor information from at least one user-facing sensor of a user device;
to obtain second sensor information from at least one environment-facing sensor of the user device; and
to process the first sensor information and the second sensor information to identify an object of user attention, the object comprising one of a physical object in an environment outside of the user device and a virtual object presented on a display screen of the user device.

### Claim 2 (depends on 1)

The apparatus of claim 1 wherein the at least one processing device comprises the user device.

### Claim 3 (depends on 1)

The apparatus of claim 1 wherein the at least one processing device comprises a cloud-based processing device configured to communicate with the user device over a network.

### Claim 4 (depends on 1)

The apparatus of claim 1 wherein the user device comprises a laptop computer, with the user-facing sensor being arranged on a display screen side of a cover of the laptop computer and the environment-facing sensor being arranged on an opposite side of the cover relative to the display screen side.

### Claim 5 (depends on 1)

The apparatus of claim 1 wherein one or more of the at least one user-facing sensor and the at least one environment-facing sensor comprise respective cameras.

### Claim 6 (depends on 1)

The apparatus of claim 1 wherein processing the first sensor information and the second sensor information to identify an object of user attention comprises:
tracking a line of sight of the user based at least in part on the first sensor information;
determining a location of the physical object in the environment outside of the user device based at least in part on the second sensor information; and
determining whether the line of sight of the user intersects with the location of the physical object in the environment outside of the user device or a location of the virtual object presented on a display screen of the user device.

### Claim 7 (depends on 1)

The apparatus of claim 1 wherein processing the first sensor information and the second sensor information to identify an object of user attention comprises:
determining a gaze vector of the user based at least in part on the first sensor information; and
determining whether or not a user gaze characterized by the gaze vector falls within designated boundaries of the display screen of the user device.

### Claim 8 (depends on 7)

The apparatus of claim 7 further comprising determining a relative positioning of the user and the user device, and determining the gaze vector based at least in part on the relative positioning.

### Claim 9 (depends on 7)

The apparatus of claim 7 wherein responsive to the user gaze characterized by the gaze vector being within designated boundaries of the display screen of the user device, determining coordinates of the user gaze and identifying the virtual object presented on a display screen of the user device based at least in part on the determined coordinates.

### Claim 10 (depends on 7)

The apparatus of claim 7 wherein responsive to the user gaze characterized by the gaze vector not being within designated boundaries of the display screen of the user device, computing current locations of respective ones of a plurality of physical elements in the environment outside the user device, detecting intersection of the gaze vector with at least one of the physical elements, and identifying the physical object in the environment outside of the user device based at least in part on the detected intersection.

### Claim 11 (depends on 10)

The apparatus of claim 10 wherein detecting intersection of the gaze vector with at least one of the physical elements comprises generating a list of potential elements and respective confidence measures for the potential elements, and wherein the physical object is identified based at least on part on one or more of the confidence measures.

### Claim 12 (depends on 1)

The apparatus of claim 1 wherein the at least one processing device is further configured to initiate performance of at least one automated action based at least in part on the identifying of the object of user attention.

### Claim 13 (depends on 12)

The apparatus of claim 12 wherein the automated action comprises automatically presenting information on the display screen of the user device relating to an identified object in the environment outside of the user device.

### Claim 14 (depends on 12)

The apparatus of claim 12 wherein the automated action comprises establishing a network connection with an additional device corresponding to an identified object in the environment outside of the user device.

### Claim 15

A computer program product comprising a non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code when executed by at least one processing device causes the at least one processing device:
to obtain first sensor information from at least one user-facing sensor of a user device;
to obtain second sensor information from at least one environment-facing sensor of the user device; and
to process the first sensor information and the second sensor information to identify an object of user attention, the object comprising one of a physical object in an environment outside of the user device and a virtual object presented on a display screen of the user device.

### Claim 16 (depends on 15)

The computer program product of claim 15 wherein processing the first sensor information and the second sensor information to identify an object of user attention comprises:
tracking a line of sight of the user based at least in part on the first sensor information;
determining a location of the physical object in the environment outside of the user device based at least in part on the second sensor information; and
determining whether the line of sight of the user intersects with the location of the physical object in the environment outside of the user device or a location of the virtual object presented on a display screen of the user device.

### Claim 17 (depends on 15)

The computer program product of claim 15 wherein processing the first sensor information and the second sensor information to identify an object of user attention comprises:
determining a gaze vector of the user based at least in part on the first sensor information; and
determining whether or not a user gaze characterized by the gaze vector falls within designated boundaries of the display screen of the user device.

### Claim 18

A method comprising:
obtaining first sensor information from at least one user-facing sensor of a user device;
obtaining second sensor information from at least one environment-facing sensor of the user device; and
processing the first sensor information and the second sensor information to identify an object of user attention, the object comprising one of a physical object in an environment outside of the user device and a virtual object presented on a display screen of the user device;
wherein the method is performed by at least one processing device comprising a processor coupled to a memory.

### Claim 19 (depends on 18)

The method of claim 18 wherein processing the first sensor information and the second sensor information to identify an object of user attention comprises:
tracking a line of sight of the user based at least in part on the first sensor information;
determining a location of the physical object in the environment outside of the user device based at least in part on the second sensor information; and
determining whether the line of sight of the user intersects with the location of the physical object in the environment outside of the user device or a location of the virtual object presented on a display screen of the user device.

### Claim 20 (depends on 18)

The method of claim 18 wherein processing the first sensor information and the second sensor information to identify an object of user attention comprises:
determining a gaze vector of the user based at least in part on the first sensor information; and
determining whether or not a user gaze characterized by the gaze vector falls within designated boundaries of the display screen of the user device.
