# US 11,443,455 B2 · Prior informed pose and scale estimation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,443,455 B2
- Document ID: US11443455B2
- Status: Granted
- Kind: B2
- Application number: 16/744,068
- Filed: 2020-01-15
- Published: 2021-04-29
- Granted: 2022-09-13
- Earliest claimed: 2019-10-24
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Victor M. Fragoso Rojas; Mei Chen; Gabriel Takacs
- Web page: https://patentgrep.com/patent/US11443455B2

## CPC

- G06T 7/75: {involving models}
- G06T 7/60: Analysis of geometric attributes
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06T 7/80: Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
- G06T 17/00: Three-dimensional [3D] modelling for computer graphics
- H04N 23/90: Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
- G06T 2207/30244: Camera pose
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A scale and pose estimation method for a camera system is disclosed. Camera data for a scene acquired by the camera system is received. A rotation prior parameter characterizing a gravity direction is received. A scale prior parameter characterizing scale of the camera system is received. A cost of a cost function is calculated for a similarity transformation that is configured to encode a scale and pose of the camera system. The cost of the cost function is influenced by the rotation prior parameter and the scale prior parameter. A solved similarity transformation is determined upon calculating a cost for the cost function that is less than a threshold cost. An estimated scale and pose of the camera system is output based on the solved similarity transformation.

## Claims

### Claim 1

A scale and pose estimation method for a camera system, the method comprising:
receiving camera data for a scene acquired by one or more cameras of the camera system;
receiving a rotation prior parameter characterizing a gravity direction;
receiving a scale prior parameter characterizing scale of the camera system;
calculating a cost of a cost function for a similarity transformation that is configured to encode a scale and pose of the camera system, such calculation being performed to optimize a rotation term and a translation term of the similarity transformation, where the cost of the cost function is selectively influenced by the rotation prior parameter and the scale prior parameter;
determining a solved similarity transformation upon calculating a cost for the cost function that is less than a threshold cost; and
outputting an estimated scale and pose of the camera system based on the solved similarity transformation.

### Claim 2 (depends on 1)

The method of claim 1, wherein the threshold cost is a minimized cost for the cost function.

### Claim 3 (depends on 1)

The method of claim 1, wherein the rotation prior parameter and the scale prior parameter selectively influence the cost of the cost function by selectively imposing a penalty on the cost function.

### Claim 4 (depends on 1)

The method of claim 1, wherein the rotation prior parameter and the scale prior parameter are derived from measurements of one or more inertial sensors.

### Claim 5 (depends on 4)

The method of claim 4, wherein the cost function includes a rotation weight corresponding to the rotation prior parameter and a scale weight corresponding to the scale prior parameter.

### Claim 6 (depends on 5)

The method of claim 5, wherein the rotation weight and the scale weight are adjusted based on sensor noise of the one or more inertial sensors.

### Claim 7 (depends on 6)

The method of claim 6, wherein the rotation weight and the scale weight decrease as sensor noise increases such that the corresponding rotation prior parameter and the corresponding scale prior parameter have less influence on the cost function as sensor noise increases, and wherein the rotation weight and the scale weight increase as sensor noise decreases such that the corresponding rotation prior parameter and the corresponding scale prior parameter have more influence on the cost function as sensor noise decreases.

### Claim 8 (depends on 6)

The method of claim 6, wherein the rotation weight and the scale weight are set to zero based on sensor noise being greater than a threshold noise level such that the corresponding rotation prior parameter and the corresponding scale prior parameter have no influence on the cost function.

### Claim 9 (depends on 1)

The method of claim 1, wherein the camera data includes a collection of three-dimensional (3D) points of a 3D model of the scene.

### Claim 10 (depends on 1)

The method of claim 1, wherein the camera system includes a plurality of different cameras having different positions fixed relative to each other.

### Claim 11 (depends on 1)

The method of claim 1, wherein the camera system includes a single camera movable throughout the scene.

### Claim 12

A computing system comprising:
one or more logic machines;
one or more storage machines holding instructions executable by the one or more logic machines to:
receive camera data for a scene acquired by one or more cameras of a camera system;
receive a rotation prior parameter characterizing a gravity direction;
receive a scale prior parameter characterizing scale of the camera system;
calculate a cost of a cost function for a similarity transformation that is configured to encode a scale and pose of the camera system, such calculation being performed to optimize a rotation term and a translation term of the similarity transformation, where the cost of the cost function is selectively influenced by the rotation prior parameter and the scale prior parameter;
determine a solved similarity transformation upon calculating a cost for the cost function that is less than a threshold cost; and
output an estimated scale and pose of the camera system based on the solved similarity transformation.

### Claim 13 (depends on 12)

The computing system of claim 12, wherein the threshold cost is a minimized cost for the cost function.

### Claim 14 (depends on 12)

The computing system of claim 12, wherein the rotation prior parameter and the scale prior parameter selectively influence the cost of the cost function by selectively imposing a penalty on the cost of the cost function.

### Claim 15 (depends on 12)

The computing system of claim 12, wherein the rotation prior parameter and the scale prior parameter are derived from measurements of one or more inertial sensors.

### Claim 16 (depends on 15)

The computing system of claim 15, wherein the cost function includes a rotation weight corresponding to the rotation prior parameter and a scale weight corresponding to the scale prior parameter.

### Claim 17 (depends on 16)

The computing system of claim 16, wherein the rotation weight and the scale weight are adjusted based on sensor noise of the one or more inertial sensors.

### Claim 18 (depends on 17)

The computing system of claim 17, wherein the rotation weight and the scale weight decrease as sensor noise increases such that the corresponding rotation prior parameter and the corresponding scale prior parameter have less influence on the cost function as sensor noise increases, and wherein the rotation weight and the scale weight increase as sensor noise decreases such that the corresponding rotation prior parameter and the corresponding scale prior parameter have more influence on the cost function as sensor noise decreases.

### Claim 19 (depends on 17)

The computing system of claim 17, wherein the rotation weight and the scale weight are set to zero based on sensor noise being greater than a threshold noise level such that the corresponding rotation prior parameter and the corresponding scale prior parameter have no influence on the cost function.

### Claim 20

A scale and pose estimation method for a camera system, the method comprising:
receiving camera data for a scene acquired by one or more cameras of the camera system;
receiving a rotation prior parameter characterizing a gravity direction;
receiving a scale prior parameter characterizing scale of the camera system, wherein the rotation prior parameter and the scale prior parameter are derived from measurements of one or more inertial sensors;
calculating a cost function for a similarity transformation that is configured to encode a scale and pose of the camera system, such calculation being performed to optimize a rotation term and a translation term of the similarity transformation, where the cost of the cost function is selectively influenced by the rotation prior parameter, a rotation weight corresponding to the rotation prior parameter, the scale prior parameter and a scale weight corresponding to the scale prior parameter;
adjusting the rotation weight and the scale weight based on the sensor noise of the one or more inertial sensors;
determining a solved similarity transformation upon calculating a cost for the cost function that is less than a threshold cost; and
outputting an estimated scale and pose of the camera system based on the solved similarity transformation.