# US 2026/0073529 A1 · Temporally sparse scale estimation for object tracking

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0073529 A1
- Document ID: US20260073529A1
- Status: Published application
- Kind: A1
- Application number: 18/883,419
- Filed: 2024-09-12
- Published: 2026-03-12
- Earliest claimed: 2024-09-12
- Assignees: Snap Inc.
- Applicants: Snap Inc.
- Inventors: Blake Lucas; Thomas Muttenthaler; Thomas Faeulhammer
- Web page: https://patentgrep.com/patent/US20260073529A1

## CPC

- G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51)
- G06V 40/28: {Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113)}
- G06T 7/60: Analysis of geometric attributes
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06T 7/80: Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
- G06T 17/00: Three-dimensional [3D] modelling for computer graphics
- G06T 2207/30196: Human being; Person
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Examples in the present disclosure relate to temporally sparse scale estimation for object tracking. A computing device detects a current orientation of an object. The computing device determines that a difference between the current orientation and at least one of a plurality of previously detected orientations of the object is less than a threshold value. Each previously detected orientation has a respective scale estimate. In response to determining that the difference is less than the threshold value, the computing device generates an effective scale estimate for the object based on a combination of the respective scale estimates for the plurality of previously detected orientations. Each respective scale estimate contributes to the effective scale estimate according to a respective difference between the current orientation and the previously detected orientation for the respective scale estimate. The computing device tracks a pose of the object based on the effective scale estimate.

## Claims

### Claim 1

A method for facilitating object tracking, the method performed by a computing device and comprising:
capturing, via one or more cameras of the computing device, at least one image of an object;
detecting a current orientation of the object based on the at least one image;
determining that a difference between the current orientation and at least one of a plurality of previously detected orientations of the object is less than a threshold value, each previously detected orientation of the plurality of previously detected orientations being associated with a respective scale estimate;
in response to determining that the difference is less than the threshold value, generating an effective scale estimate for the object based on a combination of the respective scale estimates associated with the plurality of previously detected orientations, each respective scale estimate contributing to the effective scale estimate according to a respective difference between the current orientation and the previously detected orientation associated with the respective scale estimate; and
tracking a pose of the object based on the effective scale estimate.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
processing the at least one image of the object to obtain two-dimensional (2D) landmarks associated with the object; and
processing the 2D landmarks to generate three-dimensional (3D) landmarks associated with the object, wherein the current orientation of the object is detected based on the 3D landmarks.

### Claim 3 (depends on 2)

The method of claim 2, wherein the 2D landmarks are first 2D landmarks, the 3D landmarks are first 3D landmarks, the current orientation is a first orientation detected for a first point in time, and the method further comprises:
obtaining second 2D landmarks associated with the object;
processing the second 2D landmarks to obtain second 3D landmarks associated with the object;
detecting, for a second point in time and based on the second 3D landmarks, a second orientation of the object that differs from the first orientation;
determining that a difference between the second orientation and each respective previously detected orientation of the plurality of previously detected orientations meets or exceeds the threshold value;
in response to determining that the difference between the second orientation and each respective previously detected orientation of the plurality of previously detected orientations meets or exceeds the threshold value, triggering commencement of a calibration operation to obtain a new scale estimate without utilizing the respective scale estimates associated with the plurality of previously detected orientations; and
further tracking the pose of the object based on the new scale estimate.

### Claim 4 (depends on 3)

The method of claim 3, wherein the one or more cameras comprise a plurality of cameras, and the calibration operation is performed in a multi-camera mode.

### Claim 5 (depends on 4)

The method of claim 4, wherein the tracking of the pose of the object based on the effective scale estimate is performed in a single-camera mode, the method further comprising:
automatically switching from the single-camera mode to the multi-camera mode to perform the calibration operation; and
automatically switching from the multi-camera mode back to the single-camera mode after the calibration operation to perform the further tracking of the pose of the object based on the new scale estimate in the single-camera mode.

### Claim 6 (depends on 4)

The method of claim 4, wherein the second 3D landmarks are normalized 3D landmarks, the second 2D landmarks are associated with a first camera perspective, and the calibration operation comprises obtaining the new scale estimate by:
obtaining further 2D landmarks from another camera perspective; and
minimizing a reprojection distance for estimated true locations of the second 3D landmarks in relation to the second 2D landmarks and the further 2D landmarks.

### Claim 7 (depends on 6)

The method of claim 6, wherein the new scale estimate comprises a distance between at least two estimated true locations of two of the second 3D landmarks.

### Claim 8 (depends on 3)

The method of claim 3, wherein the plurality of previously detected orientations is temporarily stored in a time buffer that is dynamically updated over time, the method further comprising:
associating the new scale estimate with the second orientation; and
updating the time buffer to include the new scale estimate.

### Claim 9 (depends on 1)

The method of claim 1, wherein the effective scale estimate comprises a weighted average of the respective scale estimates associated with the plurality of previously detected orientations.

### Claim 10 (depends on 9)

The method of claim 9, wherein a weight of each respective scale estimate within the weighted average is based on a respective angular difference between the current orientation and the previously detected orientation associated with the respective scale estimate.

### Claim 11 (depends on 10)

The method of claim 10, wherein the generating of the effective scale estimate comprises determining the weight of each respective scale estimate based on a monotonically decreasing weighting function.

### Claim 12 (depends on 1)

The method of claim 1, wherein the plurality of previously detected orientations is temporarily stored in a time buffer that is dynamically updated over time.

### Claim 13 (depends on 12)

The method of claim 12, wherein each of the plurality of previously detected orientations was detected within a predetermined time window associated with the time buffer prior to the detecting of the current orientation.

### Claim 14 (depends on 2)

The method of claim 2, wherein the detecting of the current orientation comprises:
fitting a plane to at least a subset of the 3D landmarks;
determining an orientation of the plane; and
applying the orientation of the plane as the current orientation.

### Claim 15 (depends on 2)

The method of claim 2, wherein the 3D landmarks are normalized 3D landmarks, and the tracking of the pose of the object based on the effective scale estimate comprises applying the effective scale estimate to the normalized 3D landmarks to obtain the pose of the object.

### Claim 16 (depends on 2)

The method of claim 2, wherein the processing of the at least one image and the processing of the 3D landmarks comprise executing at least one machine learning model.

### Claim 17 (depends on 1)

The method of claim 1, wherein the object is a hand of a person, and the effective scale estimate comprises at least one bone length estimate associated with the hand.

### Claim 18 (depends on 1)

The method of claim 1, wherein the computing device comprises an extended reality (XR) device, the method further comprising:
generating, by the XR device, virtual content;
determining positioning of the virtual content relative to the object based on the tracking of the pose of the object; and
causing presentation of the virtual content according to the determined positioning.

### Claim 19

An extended reality (XR) device comprising:
at least one processor; and
at least one memory storing instructions that, when executed by the at least one processor, cause the XR device to perform operations comprising:
capturing, via one or more cameras of the XR device, at least one image of an object;
detecting a current orientation of the object based on the at least one image;
determining that a difference between the current orientation and at least one of a plurality of previously detected orientations of the object is less than a threshold value, each previously detected orientation of the plurality of previously detected orientations being associated with a respective scale estimate;
in response to determining that the difference is less than the threshold value, generating an effective scale estimate for the object based on a combination of the respective scale estimates associated with the plurality of previously detected orientations, each respective scale estimate contributing to the effective scale estimate according to a respective difference between the current orientation and the previously detected orientation associated with the respective scale estimate; and
tracking a pose of the object based on the effective scale estimate.

### Claim 20

One or more non-transitory computer-readable storage media, the one or more non-transitory computer-readable storage media including instructions that, when executed by at least one processor, cause the at least one processor to perform operations comprising:
obtaining, via one or more cameras, at least one image of an object;
detecting a current orientation of the object based on the at least one image;
determining that a difference between the current orientation and at least one of a plurality of previously detected orientations of the object is less than a threshold value, each previously detected orientation of the plurality of previously detected orientations being associated with a respective scale estimate;
in response to determining that the difference is less than the threshold value, generating an effective scale estimate for the object based on a combination of the respective scale estimates associated with the plurality of previously detected orientations, each respective scale estimate contributing to the effective scale estimate according to a respective difference between the current orientation and the previously detected orientation associated with the respective scale estimate; and
tracking a pose of the object based on the effective scale estimate.
