# US 2024/0005547 A1 · Object pose tracking from video images

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0005547 A1
- Document ID: US20240005547A1
- Status: Published application
- Kind: A1
- Application number: 17/750,785
- Filed: 2022-05-23
- Published: 2024-01-04
- Earliest claimed: 2022-05-23
- Assignees: NVIDIA Corporation
- Applicants: NVIDIA Corporation
- Inventors: Yunzhi Lin; Jonathan Tremblay; Stephen Walter Tyree; Stanley Thomas Birchfield
- Web page: https://patentgrep.com/patent/US20240005547A1

## CPC

- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06T 7/277: involving stochastic approaches, e.g. using Kalman filters
- G06T 2207/10016: Video; Image sequence
- G06T 2207/10024: Color image
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Apparatuses, systems, and techniques to determined a pose of an object from a plurality of images. In at least one embodiment, the pose of an object is determined from at least two images of a video sequence using one or more neural networks, in which the neural network produces a distribution of pose information that is filtered to determine the current pose.

## Claims

### Claim 1

A computer-implemented method comprising:
determining a first distribution of pose information for an object in an image using a trained network from the image, a prior image, and a second distribution of prior pose information of the object in the prior image;
determining a pose of the object in the image by at least filtering the first distribution of pose information; and
providing the pose.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, wherein the pose is a 6-degree of freedom pose.

### Claim 3 (depends on 1)

The computer-implemented method of claim 1, wherein the image and the prior image are two-dimensional images.

### Claim 4 (depends on 1)

The computer-implemented method of claim 1, wherein the pose is determined by applying a Bayesian filter to the first distribution of pose information.

### Claim 5 (depends on 1)

The computer-implemented method of claim 1, wherein the first distribution of pose information and the second distribution of prior pose information includes a center heatmap and a keypoint heatmap.

### Claim 6 (depends on 1)

The computer-implemented method of claim 1, wherein the network is used to calculate a center track offset and a keypoint track offset.

### Claim 7 (depends on 1)

The computer-implemented method of claim 1, wherein the pose information defines a bounding cuboid for the object.

### Claim 8 (depends on 1)

The computer-implemented method of claim 1, wherein the network determines the first distribution of pose information using a third distribution of additional pose information older than the prior pose information.

### Claim 9 (depends on 1)

The computer-implemented method of claim 1, wherein an uncertainty is determined by applying a Kalman filter to the first distribution of pose information.

### Claim 10

A system comprising one or more circuits to:
determine a first distribution of pose information for an object in an image using a trained network from the image, a prior image, and a second distribution of prior pose information of the object in the prior image;
determine a pose of the object in the image by at least filtering the first distribution of pose information; and
provide the pose.

### Claim 11 (depends on 10)

The system of claim 10, wherein the pose is a 6-degree of freedom pose.

### Claim 12 (depends on 10)

The system of claim 10, wherein the image and the prior image are two-dimensional images.

### Claim 13 (depends on 10)

The system of claim 10, wherein the pose is determined by applying a Bayesian filter to the first distribution of pose information.

### Claim 14 (depends on 10)

The system of claim 10, wherein the first distribution of pose information and the second distribution of prior pose information includes a center heatmap and a keypoint heatmap.

### Claim 15 (depends on 10)

The system of claim 10, wherein the network is used to calculate a center track offset and a keypoint track offset.

### Claim 16 (depends on 10)

The system of claim 10, wherein the pose information defines a bounding cuboid for the object.

### Claim 17 (depends on 10)

The system of claim 10, wherein the network determines the first distribution of pose information using a third distribution of additional pose information older than the prior pose information.

### Claim 18 (depends on 10)

The system of claim 10, wherein an uncertainty is determined by applying a Kalman filter to the first distribution of pose information.

### Claim 19

A machine-readable medium having stored thereon a set of instructions, which if performed by one or more processors, cause the one or more processors to at least:
determine a first distribution of pose information for an object in an image using a trained network from the image, a prior image, and a second distribution of prior pose information of the object in the prior image;
determine a pose of the object in the image by at least filtering the first distribution of pose information; and
provide the pose.

### Claim 20 (depends on 19)

The machine-readable medium of claim 19, wherein the pose is a 6-degree of freedom pose.

### Claim 21 (depends on 19)

The machine-readable medium of claim 19, wherein the image and the prior image are two-dimensional images.

### Claim 22 (depends on 19)

The machine-readable medium of claim 19, wherein the pose is determined by applying a Bayesian filter to the first distribution of pose information.

### Claim 23 (depends on 19)

The machine-readable medium of claim 19, wherein the first distribution of pose information and the second distribution of prior pose information includes a center heatmap and a keypoint heatmap.

### Claim 24 (depends on 19)

The machine-readable medium of claim 19, wherein the network is used to calculate a center track offset and a keypoint track offset.

### Claim 25 (depends on 19)

The machine-readable medium of claim 19, wherein the pose information defines a bounding cuboid for the object.

### Claim 26 (depends on 19)

The machine-readable medium of claim 19, wherein the network determines the first distribution of pose information using a third distribution of additional pose information older than the prior pose information.

### Claim 27 (depends on 19)

The machine-readable medium of claim 19, wherein an uncertainty is determined by applying a Kalman filter to the first distribution of pose information.
