# US 2026/0073551 A1 · Non-user hand rejection for extended reality devices

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0073551 A1
- Document ID: US20260073551A1
- Status: Published application
- Kind: A1
- Application number: 18/828,718
- Filed: 2024-09-09
- Published: 2026-03-12
- Earliest claimed: 2024-09-09
- Assignees: Snap Inc.
- Applicants: Snap Inc.
- Inventors: Thomas Faeulhammer; Balázs Tóth; Daniel Wolf
- Web page: https://patentgrep.com/patent/US20260073551A1

## CPC

- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G02B 27/017: {Head mounted}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/0304: {Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042)}
- G06T 7/11: Region-based segmentation
- G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51)
- G06V 20/20: in augmented reality scenes
- G06V 40/107: {Static hand or arm}
- G06V 40/28: {Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113)}
- G02B 2027/0138: {comprising image capture systems, e.g. camera}
- G02B 2027/014: {comprising information/image processing systems}
- G06T 2207/30196: Human being; Person
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Examples in the present disclosure relate to systems and methods for detecting and rejecting a non-user hand in the context of egocentric hand tracking performed by an extended reality (XR) device. While the XR device is worn by a user, the XR device captures at least one image of a hand and processes the at least one image to detect the hand. After detecting the hand, the XR device determines positioning of the hand relative to the XR device or another object in a field of view of the XR device. The XR device detects that the hand is a non-user hand. In response to detecting that the hand is a non-user hand, the XR device excludes the non-user hand from the egocentric hand tracking such that the non-user hand is not tracked for the user.

## Claims

### Claim 1

An extended reality (XR) device comprising:
one or more optical sensors;
one or more processors; and
at least one memory storing instructions that, when executed by the one or more processors, cause the XR device, when worn by a user, to perform operations comprising:
capturing, via the one or more optical sensors, at least one image of a hand;
processing the at least one image to detect the hand, the processing comprising executing at least one object detection machine learning model that returns a confidence value;
after detecting the hand, determining positioning of the hand relative to at least one of the XR device or an additional object in a field of view of the XR device, the determining of the positioning of the hand relative to at least one of the XR device or the additional object being triggered based on determining that the confidence value meets or exceeds a threshold;
detecting, based on the positioning of the hand relative to at least one of the XR device or the additional object, that the hand is a non-user hand; and
in response to detecting that the hand is the non-user hand, excluding the hand from egocentric hand tracking performed by the XR device with respect to the user.

### Claim 2 (depends on 1)

The XR device of claim 1, wherein the hand is detected to be the non-user hand based on both the positioning of the hand relative to the XR device and the positioning of the hand relative to the additional object in the field of view of the XR device.

### Claim 3 (depends on 1)

The XR device of claim 1, wherein the detecting, based on the positioning of the hand relative to at least one of the XR device or the additional object, that the hand is the non-user hand, comprises executing a plurality of rejection filters in a predetermined sequence.

### Claim 4 (depends on 1)

The XR device of claim 1, wherein the determining of the positioning of the hand relative to at least one of the XR device or the additional object comprises:
generating a three-dimensional (3D) position associated with the hand; and
determining, based on the 3D position associated with the hand, a distance between the hand and at least one of the user or the XR device, wherein the hand is detected to be the non-user hand based on the distance meeting or exceeding a threshold.

### Claim 5 (depends on 1)

The XR device of claim 1, wherein the determining of the positioning of the hand relative to at least one of the XR device or the additional object comprises:
generating a zone associated with a location of the hand within the at least one image, wherein the hand is detected to be the non-user hand based on a size of the zone satisfying a predetermined condition.

### Claim 6 (depends on 5)

The XR device of claim 5, wherein the generating of the zone comprises generating a bounding element that covers at least part of the hand, and the size of the zone comprises a two-dimensional (2D) area of the bounding element.

### Claim 7 (depends on 1)

The XR device of claim 1, wherein the additional object is a user hand that is being tracked using the egocentric hand tracking performed by the XR device, and the determining of the positioning of the hand relative to at least one of the XR device or the additional object comprises:
comparing the positioning of the hand with positioning of the user hand, wherein the hand is detected to be the non-user hand based on the positioning of the hand relative to the user hand being invalid according to a predetermined condition.

### Claim 8 (depends on 7)

The XR device of claim 7, wherein the predetermined condition indicates, based on a chirality of the user hand, on which side of the user hand the hand is to appear in the at least one image.

### Claim 9 (depends on 1)

The XR device of claim 1, the operations further comprising:
identifying a chirality of the hand, wherein the hand is detected to be the non-user hand based on both the chirality of the hand and horizontal positioning of the hand within a scene captured by the at least one image.

### Claim 10 (depends on 9)

The XR device of claim 9, wherein the horizontal positioning of the hand is provided as input to a decision function to generate a value indicative of a likelihood that the positioning of the hand is invalid.

### Claim 11 (depends on 1)

The XR device of claim 1, wherein the determining of the positioning of the hand relative to at least one of the XR device or the additional object comprises:
determining an entry region of the hand within the field of view of the XR device, wherein the hand is detected to be the non-user hand based on the entry region being invalid according to a predetermined condition.

### Claim 12 (depends on 1)

The XR device of claim 1, wherein the additional object comprises an arm that appears in the at least one image, and the determining of the positioning of the hand relative to at least one of the XR device or the additional object comprises:
detecting that the arm corresponds to the hand, wherein the hand is detected to be the non-user hand based on positioning of the arm relative to at least one of the hand or the XR device.

### Claim 13 (depends on 1)

The XR device of claim 1, the operations further comprising:
causing presentation, to the user, of a gesture-driven user interface comprising virtual content; and
performing the egocentric hand tracking to obtain, from the user, user input for navigation of the gesture-driven user interface.

### Claim 14 (depends on 1)

The XR device of claim 1, wherein, for a given hand detected by the XR device during a detection phase, the egocentric hand tracking is performed in a tracking phase that follows completion of the detection phase.

### Claim 15 (depends on 14)

The XR device of claim 14, wherein the excluding of the hand from the egocentric hand tracking is performed after completion of the detection phase for the hand.

### Claim 16 (depends on 14)

The XR device of claim 14, wherein the detecting, based on the positioning of the hand relative to at least one of the XR device or the additional object, that the hand is a non-user hand, comprises executing a plurality of rejection filters in a predetermined sequence that comprises at least one rejection filter that is executed before commencement of the tracking phase and at least one further rejection filter that is executed during the tracking phase.

### Claim 17

(canceled)

### Claim 18 (depends on 1)

The XR device of claim 1, wherein the XR device is a head-wearable XR device, and the operations are performed while the XR device is worn on a head of the user.

### Claim 19

A method performed by an extended reality (XR) device while the XR device is worn by a user, the method comprising:
capturing, via one or more optical sensors, at least one image of a hand;
processing the at least one image to detect the hand, the processing comprising executing at least one object detection machine learning model that returns a confidence value;
after detecting the hand, determining positioning of the hand relative to at least one of the XR device or an additional object in a field of view of the XR device, the determining of the positioning of the hand relative to at least one of the XR device or the additional object being triggered based on determining that the confidence value meets or exceeds a threshold;
detecting, based on the positioning of the hand relative to at least one of the XR device or the additional object, that the hand is a non-user hand; and
in response to detecting that the hand is the non-user hand, excluding the hand from egocentric hand tracking performed by the XR device with respect to the user.

### Claim 20

One or more non-transitory computer-readable storage media, the one or more non-transitory computer-readable storage media including instructions that, when executed by at least one processor of an extended reality (XR) device worn by a user, cause the XR device to perform operations comprising:
capturing, via one or more optical sensors, at least one image of a hand;
processing the at least one image to detect the hand, the processing comprising executing at least one object detection machine learning model that returns a confidence value;
after detecting the hand, determining positioning of the hand relative to at least one of the XR device or an additional object in a field of view of the XR device, the determining of the positioning of the hand relative to at least one of the XR device or the additional object being triggered based on determining that the confidence value meets or exceeds a threshold;
detecting, based on the positioning of the hand relative to at least one of the XR device or the additional object, that the hand is a non-user hand; and
in response to detecting that the hand is the non-user hand, excluding the hand from egocentric hand tracking performed by the XR device with respect to the user.

### Claim 21 (depends on 19)

The method of claim 19, wherein the hand is detected to be the non-user hand based on both the positioning of the hand relative to the XR device and the positioning of the hand relative to the additional object in the field of view of the XR device.
