# US 2020/0349341 A1 · Performing an iterative bundle adjustment for an imaging device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0349341 A1
- Document ID: US20200349341A1
- Status: Published application
- Kind: A1
- Application number: 16/399,519
- Filed: 2019-04-30
- Published: 2020-11-05
- Earliest claimed: 2019-04-30
- Assignees: Samsung Electronics Co., Ltd.
- Inventors: Manish Goel; Yuming Zhu
- Web page: https://patentgrep.com/patent/US20200349341A1

## CPC

- G06T 1/20: Processor architectures; Processor configuration, e.g. pipelining
- G06T 7/579: from motion
- G06T 7/73: using feature-based methods
- G06V 10/94: Hardware or software architectures specially adapted for image or video understanding
- G06V 20/64: Three-dimensional [3D] objects
- G06T 2207/30244: Camera pose
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A method of performing an iterative bundle adjustment for an imaging device is described. The method comprising implementing a plurality of functions in performing a bundle adjustment. Predetermined functions of the plurality of functions may be started using a processor for a second iteration in parallel with a first iteration of the plurality of functions. A result of the predetermined functions started during the first iteration may be used in a second iteration. An output of the bundle adjustment may then be generated for successive iterations.

## Claims

### Claim 1

A method of performing an iterative bundle adjustment for an imaging device, the method comprising:
implementing a plurality of functions in performing a bundle adjustment;
starting, using a processor, predetermined functions of the plurality of functions for a second iteration of the plurality of functions in parallel with a first iteration of the plurality of functions;
using a result of the predetermined functions started during the first iteration in the second iteration; and
generating an output of the bundle adjustment for successive iterations.

### Claim 2 (depends on 1)

The method of claim 1, wherein the plurality of functions comprises a Jacobian function, a Hessian function, a solve function, and a cost function.

### Claim 3 (depends on 2)

The method of claim 2, wherein the predetermined functions started for a second iteration in parallel with the first iteration comprise the Jacobian function and the Hessian function.

### Claim 4 (depends on 2)

The method of claim 2, wherein a previous valid Jacobian and Hessian solution is used for the second iteration when the second iteration fails.

### Claim 5 (depends on 1)

The method of claim 1, wherein intermediate results of the predetermined functions are saved and reused across iterations of performing the bundle adjustment.

### Claim 6 (depends on 1)

The method of claim 1, wherein starting predetermined functions of the plurality of functions for a second iteration in parallel comprises performing the predetermined functions on a plurality of branches.

### Claim 7 (depends on 6)

The method of claim 6 wherein a number of the plurality of branches performing the predetermined functions can be adjusted in run-time.

### Claim 8 (depends on 7)

The method of claim 7, further comprising providing iterative control based upon results of a cost function for all branches of the plurality of branches.

### Claim 9

An imaging device performing an iterative bundle adjustment, the imaging device comprising:
a processor configured to:
implement a plurality of functions in performing a bundle adjustment;
start predetermined functions of the plurality of functions for a second iteration in parallel with a first iteration of the plurality of functions;
use the result of the predetermined functions started during the first iteration in the second iteration; and
generate an output of the bundle adjustment for successive iterations.

### Claim 10 (depends on 9)

The imaging device of claim 9, wherein the plurality of functions comprises a Jacobian function, a Hessian function, a solve function, and a cost function.

### Claim 11 (depends on 10)

The imaging device of claim 10, wherein the predetermined functions started for a second iteration in parallel with the first iteration comprise the Jacobian function and the Hessian function.

### Claim 12 (depends on 10)

The imaging device of claim 10, wherein a previous valid Jacobian and Hessian solution is used for the second iteration when the second iteration fails.

### Claim 13 (depends on 9)

The imaging device of claim 9, wherein intermediate results of the predetermined functions are saved and reused across iterations of performing the bundle adjustment.

### Claim 14 (depends on 9)

The imaging device of claim 9, wherein starting predetermined functions of the plurality of functions for a second iteration in parallel comprises performing the predetermined functions on a plurality of branches.

### Claim 15 (depends on 14)

The imaging device of claim 14 wherein a number of the plurality of branches performing the predetermined functions can be adjusted in run-time.

### Claim 16 (depends on 15)

The imaging device of claim 15, further comprising providing iterative control based upon results of the cost function for all branches of the plurality of branches.

### Claim 17

A non-transitory computer-readable storage medium having data stored therein representing software executable by a computer for performing an iterative bundle adjustment, the non-transitory computer-readable storage medium comprising:
implementing a plurality of functions in performing a bundle adjustment;
starting predetermined functions of the plurality of functions for a second iteration in parallel with a first iteration of the plurality of functions;
using a result of the predetermined functions started during the first iteration in the second iteration; and
generating an output of the bundle adjustment for successive iterations.

### Claim 18 (depends on 17)

The non-transitory computer-readable storage medium of claim 17, wherein plurality of functions comprises a Jacobian function, a Hessian function, a solve function, and a cost function.

### Claim 19 (depends on 18)

The non-transitory computer-readable storage medium of claim 18, wherein the predetermined functions started for a second iteration in parallel with the first iteration comprise the Jacobian function and the Hessian function.

### Claim 20 (depends on 19)

The non-transitory computer-readable storage medium of claim 19, wherein a previous valid Jacobian and Hessian solution is used for the second iteration when the second iteration fails.
