# US 2026/0073545 A1 · Hand pose-dependent scale estimation for extended reality > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2026/0073545 A1 - Document ID: US20260073545A1 - Status: Published application - Kind: A1 - Application number: 18/883,633 - Filed: 2024-09-12 - Published: 2026-03-12 - Earliest claimed: 2024-09-12 - Assignees: Snap Inc. - Applicants: Snap Inc. - Inventors: Markus Diem; Thomas Muttenthaler; Daniele Wolf; Jeremias Beyene Yehdegho - Web page: https://patentgrep.com/patent/US20260073545A1 ## CPC - G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)} - G06T 7/60: Analysis of geometric attributes - G06F 1/163: {Wearable computers, e.g. on a belt} - G06F 1/1686: {the I/O peripheral being an integrated camera} - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06F 3/0304: {Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042)} - G06T 7/246: using feature-based methods, e.g. the tracking of corners or segments - G06T 7/251: {involving models} - G06T 7/292: Multi-camera tracking - G06T 7/593: from stereo images - G06T 7/73: using feature-based methods - G06V 10/14: Optical characteristics of the device performing the acquisition or on the illumination arrangements - G06V 10/32: Normalisation of the pattern dimensions - G06V 20/20: in augmented reality scenes - G06V 40/113: {Recognition of static hand signs} - G06V 40/117: {Biometrics derived from hands} - G06T 2207/10012: Stereo images - G06T 2207/10021: Stereoscopic video; Stereoscopic image sequence - G06T 2207/20081: Training; Learning - G06T 2207/20084: Artificial neural networks [ANN] - G06T 2207/30196: Human being; Person - G06V 10/82: using neural networks - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract Examples in the present disclosure relate to scale estimation for facilitating extended reality (XR) experiences. An image of a hand of a user is captured via one or more optical sensors of an XR device. The image is processed to detect a hand pose relative to the XR device. A hand scale estimate corresponding to the detected hand pose is accessed. The hand scale estimate is one of a plurality of hand scale estimates each uniquely associated with a respective hand pose. The hand scale estimate is applied to generate positional data for one or more features of the hand of the user. The XR device tracks the hand of the user based on the positional data while the user uses the XR device. ## Claims ### Claim 1 A method comprising: capturing, via one or more optical sensors of an extended reality (XR) device, at least one image of a hand of a user of the XR device; processing the at least one image to detect a hand pose relative to the XR device; accessing one or more existing hand scale estimates, each uniquely associated with a respective hand pose; determining that the detected hand pose does not correspond to any of the one or more existing hand scale estimates; based on determining that the detected hand pose does not correspond to any of the one or more existing hand scale estimates: triggering a calibration operation to obtain a hand scale estimate associated with the detected hand pose, and updating the one or more existing hand scale estimates to add the hand scale estimate associated with the detected hand pose; applying the hand scale estimate associated with the detected hand pose to generate positional data for one or more features of the hand of the user; and tracking, by the XR device and based on the positional data, the hand of the user while the user is using the XR device. ### Claim 2 (depends on 1) The method of claim 1, wherein the detected hand pose is a first hand pose, the hand scale estimate associated with the first hand pose is a first hand scale estimate, and the method further comprises: after detecting the first hand pose, detecting that the hand of the user has assumed a second hand pose that differs from the first hand pose; determining that the second hand pose corresponds to one of the respective hand poses associated with the existing hand scale estimates; accessing, from the existing hand scale estimates, a second hand scale estimate associated with the corresponding one of the respective hand poses, the second hand scale estimate differing from the first hand scale estimate; applying the second hand scale estimate to generate further positional data for the one or more features of the hand of the user; and further tracking the hand of the user based on the further positional data. ### Claim 3 (canceled) ### Claim 4 (depends on 1) The method of claim 1, wherein the one or more optical sensors comprise a plurality of cameras, and the calibration operation is performed in a multi-camera mode. ### Claim 5 (depends on 4) The method of claim 4, wherein the tracking of the hand is performed in a single-camera mode, and the method further comprises: automatically switching from the multi-camera mode to the single-camera mode after generating of the hand scale estimate. ### Claim 6 (depends on 1) The method of claim 1, wherein the determining that the detected hand pose does not correspond to any of the one or more existing hand scale estimates comprises determining that the detected hand pose does not meet a matching threshold with respect to any the respective hand pose associated with the one or more existing hand scale estimates. ### Claim 7 (depends on 1) The method of claim 1, wherein the one or more existing hand scale estimates comprises a set of existing hand scale estimates, the method further comprising: identifying, from among the respective hand poses associated with the existing hand scale estimates in the set of existing hand scale estimates, an existing hand pose that is closest to the detected hand pose, the existing hand pose being uniquely associated with an existing hand scale estimate in the set of existing hand scale estimates; and using the existing hand scale estimate associated with the existing hand pose to facilitate the tracking of the hand while the calibration operation is in progress. ### Claim 8 (depends on 1) The method of claim 1, wherein the positional data comprises three-dimensional (3D) positions related to one or more landmarks on the hand that are tracked by the XR device. ### Claim 9 (depends on 8) The method of claim 8, wherein the positional data comprises absolute depth information related to the one or more landmarks, the applying of the hand scale estimate to generate the positional data comprising: applying the hand scale estimate to obtain the absolute depth information from relative depth information related to the one or more landmarks. ### Claim 10 (depends on 9) The method of claim 9, further comprising, before obtaining the absolute depth information: executing a neural network to generate the relative depth information based on two-dimensional (2D) positions related to the one or more landmarks. ### Claim 11 (depends on 1) The method of claim 1, wherein the hand scale estimate comprises at least one bone length estimate associated with the band. ### Claim 12 (depends on 11) The method of claim 11, wherein the calibration operation comprises: accessing stereo image data for the hand; processing the stereo image data by performing triangulation on landmarks on the hand; and measuring one or more distances between at least some of the landmarks on the hand to determine the at least one bone length estimate. ### Claim 13 (depends on 12) The method of claim 12, further comprising: repeating the calibration operation until the at least one bone length estimate meets a predetermined stabilization threshold. ### Claim 14 (depends on 1) The method of claim 1, wherein the tracking of the hand comprises providing, by the XR device, the positional data as input to a machine learning model that is executed to track the hand. ### Claim 15 (depends on 1) The method of claim 1, wherein the processing of the at least one image comprises generating one or more vectors representing the detected hand pose, the one or more vectors being generated based on relative positions of landmarks on the hand. ### Claim 16 (depends on 1) The method of claim 1, wherein each of the respective hand poses is represented by one or more respective vectors. ### Claim 17 (depends on 1) The method of claim 1, further comprising: generating, by the XR device, virtual content; determining positioning of the virtual content relative to the hand of the user based on the positional data; and causing presentation of the virtual content to the user. ### Claim 18 (depends on 1) The method of claim 1, wherein the XR device is a head-mounted XR device. ### Claim 19 An extended reality (XR) device comprising: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the XR device to perform operations comprising: capturing, via one or more optical sensors of the XR device, at least one image of a hand of a user of the XR device; processing the at least one image to detect a hand pose relative to the XR device; accessing one or more existing hand scale estimates, each uniquely associated with a respective hand pose; determining that the detected hand pose does not correspond to any of the one or more existing hand scale estimates; based on determining that the detected hand pose does not correspond to any of the one or more existing hand scale estimates: triggering a calibration operation to obtain a hand scale estimate associated with the detected hand pose, and updating the one or more existing hand scale estimates to add the hand scale estimate associated with the detected hand pose; applying the hand scale estimate associated with the detected hand pose to generate positional data for one or more features of the hand of the user; and tracking, based on the positional data, the hand of the user while the user is using the XR device. ### Claim 20 One or more non-transitory computer-readable storage media, the one or more non-transitory computer-readable storage media including instructions that, when executed by at least one processor, cause the at least one processor to perform operations comprising: capturing, via one or more optical sensors of an extended reality (XR) device, at least one image of a hand of a user of the XR device; processing the at least one image to detect a hand pose relative to the XR device; accessing one or more existing hand scale estimates, each uniquely associated with a respective hand pose; determining that the detected hand pose does not correspond to any of the one or more existing hand scale estimates; based on determining that the detected hand pose does not correspond to any of the one or more existing hand scale estimates: triggering a calibration operation to obtain a hand scale estimate associated with the detected hand pose, and updating the one or more existing hand scale estimates to add the hand scale estimate associated with the detected hand pose; applying the hand scale estimate associated with the detected hand pose to generate positional data for one or more features of the hand of the user; and tracking, based on the positional data, the hand of the user while the user is using the XR device. ### Claim 21 (depends on 19) The XR device of claim 19, wherein the one or more optical sensors comprise a plurality of cameras, the calibration operation is performed in a multi-camera mode, the tracking of the hand is performed in a single-camera mode, and the operations further comprise: automatically switching from the multi-camera mode to the single-camera mode after generating of the hand scale estimate.