# US 2022/0044441 A1 · Systems and methods for pose detection and measurement

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0044441 A1
- Document ID: US20220044441A1
- Status: Published application
- Kind: A1
- Application number: 17/420,992
- Filed: 2021-01-29
- Published: 2022-02-10
- Earliest claimed: 2020-01-29
- Assignees: Boston Polarimetrics, Inc.
- Applicants: Boston Polarimetrics, Inc.
- Inventors: Agastya KALRA; Achuta KADAMBI; Kartik VENKATARAMAN; Vage TAAMAZYAN
- Web page: https://patentgrep.com/patent/US20220044441A1

## CPC

- G06V 10/454: {Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]}
- G06T 7/579: from motion
- G06T 7/75: {involving models}
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
- G06V 10/82: using neural networks
- G06V 20/647: {by matching two-dimensional images to three-dimensional objects}
- G06T 2207/10012: Stereo images
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A method for estimating a pose of an object includes: receiving a plurality of images of the object captured from multiple viewpoints with respect to the object; initializing a current pose of the object based on computing an initial estimated pose of the object from at least one of the plurality of images; predicting a plurality of 2-D keypoints associated with the object from each of the plurality of images; and computing an updated pose that minimizes a cost function based on a plurality of differences between the 2-D keypoints and a plurality of 3-D keypoints associated with a 3-D model of the object as arranged in accordance with the current pose, and as projected to each of the viewpoints.

## Claims

### Claim 1

(canceled)

### Claim 2

(canceled)

### Claim 3

A method for estimating a pose of an object comprising:
receiving a plurality of images of the object captured from multiple viewpoints with respect to the object;
initializing a current pose of the object based on computing an initial estimated pose of the object from at least one of the plurality of images;
predicting a plurality of 2-D keypoints associated with the object from each of the plurality of images; and
computing an updated pose that minimizes a cost function based on a plurality of differences between the 2-D keypoints and a plurality of 3-D keypoints associated with a 3-D model of the object as arranged in accordance with the current pose, and as projected to each of the viewpoints,
wherein each of the plurality of differences corresponds to a different viewpoint of the viewpoints, and
wherein each of the differences is computed for a viewpoint of the viewpoints based on a difference between:
the plurality of 2-D keypoints associated with an image of the plurality of images corresponding to the viewpoint; and
projecting the 3-D keypoints of the 3-D model arranged in the current pose to the viewpoint,
wherein the cost function is:
min
R
o
,
T
o
⁢
∑
i
:
N
,
j
:
M
⁢
c
i
⁢
j
⁢

[
u
ij
,
v
i
⁢
j
]
-
K
j
⁡
[
R
j
⁢
T
j
]
⁡
[
R
o
⁢
T
o
]
⁡
[
x
i
,
y
i
,
z
i
]

wherein Ro, To is the pose of the object, i is an index iterating through N 3-D keypoints, j is an index iterating through M viewpoints, cij is a confidence for a corresponding keypoint prediction [uij, vij] of an i-th keypoint in an image for corresponding to a j-th viewpoint, Rj, Tj, Kj are extrinsic parameters and intrinsic parameters of a j-th camera used to capture an image from a j-th viewpoint, and [xi, yi, zi] is an i-th 3-D keypoint.

### Claim 4

A method for estimating a pose of an object comprising:
receiving a plurality of images of the object captured from multiple viewpoints with respect to the object;
initializing a current pose of the object based on computing an initial estimated pose of the object from at least one of the plurality of images;
predicting a plurality of 2-D keypoints associated with the object from each of the plurality of images; and
computing an updated pose that minimizes a cost function based on a plurality of differences between the 2-D keypoints and a plurality of 3-D keypoints associated with a 3-D model of the object as arranged in accordance with the current pose, and as projected to each of the viewpoints,
wherein each of the plurality of differences corresponds to a different viewpoint of the viewpoints, and
wherein each of the differences is computed for a viewpoint of the viewpoints based on a difference between:
the plurality of 2-D keypoints associated with an image of the plurality of images corresponding to the viewpoint; and
projecting the 3-D keypoints of the 3-D model arranged in the current pose to the viewpoint,
wherein the cost function further accounts for symmetries in the object in accordance with:
min
R
o
,
T
o
⁢
∑
i
:
N
,
j
:
M
⁢
min
S
∼
v
j
⁢
c
i
⁢
j
⁢

[
u
ij
,
v
i
⁢
j
]
-
K
j
⁢
S
⁡
[
R
j
⁢
T
j
]
⁡
[
R
o
⁢
T
o
]
⁡
[
x
i
,
y
i
,
z
i
]

wherein Ro, To is the pose of the object, i is an index iterating through N 3-D keypoints, j is an index iterating through M viewpoints, cij is a confidence for a corresponding keypoint prediction [uij, vij] of an i-th keypoint in an image from a j-th viewpoint, Rj, Tj, Kj are extrinsic parameters and intrinsic parameters of the j-th camera used to capture the images from a j-th viewpoint, [xi, yi, zi] is an i-th 3-D keypoint, S is a transform between different symmetries of the object, and vj is the j-th view.

### Claim at position 5

-30. (canceled)