# US 10,371,530 B2 · Systems and methods for using a global positioning system velocity in visual-inertial odometry

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,371,530 B2
- Document ID: US10371530B2
- Status: Granted
- Kind: B2
- Application number: 15/703,483
- Filed: 2017-09-13
- Published: 2018-07-05
- Granted: 2019-08-06
- Earliest claimed: 2017-01-04
- Assignees: QUALCOMM Incorporated
- Applicants: QUALCOMM Incorporated
- Inventors: Arvind Ramanandan; Murali Chari; Avdhut Joshi
- Web page: https://patentgrep.com/patent/US10371530B2

## CPC

- G01C 21/1656: {with passive imaging devices, e.g. cameras}
- G01S 19/47: the supplementary measurement being an inertial measurement, e.g. tightly coupled inertial
- G01S 19/485: {whereby the further system is an optical system or imaging system}
- G01S 19/49: whereby the further system is an inertial position system, e.g. loosely-coupled
- G01S 19/52: Determining velocity
- G01S 19/53: Determining attitude
- G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S)
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES

## Abstract

A method performed by an electronic device is described. The method includes determining a predicted velocity relative to Earth corresponding to a first epoch using a camera and an inertial measurement unit (IMU). The method also includes determining, using a Global Positioning System (GPS) receiver, a GPS velocity relative to Earth. The method further includes determining a difference vector between the predicted velocity and the GPS velocity. The method additionally includes refining a bias estimate and a scale factor estimate of IMU measurements proportional to the difference vector. The method also includes refining a misalignment estimate between the camera and the IMU based on the difference vector. The method further includes providing pose information based on the refined bias estimate, the refined scale factor, and the refined misalignment estimate.

## Claims

### Claim 1

A method performed by an electronic device, comprising:
determining a predicted velocity relative to Earth corresponding to a first epoch using a camera and an inertial measurement unit (IMU);
determining, using a Global Positioning System (GPS) receiver, a GPS velocity relative to Earth;
determining a difference vector between the predicted velocity and the GPS velocity;
refining a bias estimate and a scale factor estimate of IMU measurements proportional to the difference vector;
refining a misalignment estimate between the camera and the IMU based on the difference vector; and
providing pose information based on the refined bias estimate, the refined scale factor estimate, and the refined misalignment estimate.

### Claim 2 (depends on 1)

The method of claim 1, further comprising determining a second predicted velocity corresponding to a second epoch using the camera and the IMU based on the refined bias estimate, the refined scale factor estimate, and the refined misalignment estimate.

### Claim 3 (depends on 1)

The method of claim 1, wherein the misalignment estimate comprises a translation displacement estimate and a rotation displacement estimate.

### Claim 4 (depends on 1)

The method of claim 1, further comprising:
obtaining an error metric indicating an uncertainty in the GPS velocity; and
weighting the GPS velocity in a Kalman filter based on the error metric.

### Claim 5 (depends on 1)

The method of claim 1, wherein a predicted velocity error is bounded based on the GPS velocity.

### Claim 6 (depends on 1)

The method of claim 1, wherein the GPS velocity is utilized to reduce visual-inertial odometry (VIO) drift.

### Claim 7 (depends on 1)

The method of claim 1, further comprising rejecting at least one of a set of GPS velocities based on at least one of a set of difference vectors.

### Claim 8 (depends on 1)

The method of claim 1, further comprising rejecting at least one of a set of GPS velocities based on an error metric.

### Claim 9 (depends on 1)

The method of claim 1, wherein the pose information comprises a 6 degrees of freedom (DOF) camera pose.

### Claim 10 (depends on 1)

The method of claim 1, wherein a vehicle comprises the electronic device.

### Claim 11

An electronic device, comprising:
a memory;
a processor coupled to the memory, wherein the processor is configured to:
determine a predicted velocity relative to Earth corresponding to a first epoch using a camera and an inertial measurement unit (IMU);
determine, using a Global Positioning System (GPS) receiver, a GPS velocity relative to Earth;
determine a difference vector between the predicted velocity and the GPS velocity;
refine a bias estimate and a scale factor estimate of IMU measurements proportional to the difference vector;
refine a misalignment estimate between the camera and the IMU based on the difference vector; and
provide pose information based on the refined bias estimate, the refined scale factor estimate, and the refined misalignment estimate.

### Claim 12 (depends on 11)

The electronic device of claim 11, wherein the processor is configured to determine a second predicted velocity corresponding to a second epoch using the camera and the IMU based on the refined bias estimate, the refined scale factor estimate, and the refined misalignment estimate.

### Claim 13 (depends on 11)

The electronic device of claim 11, wherein the misalignment estimate comprises a translation displacement estimate and a rotation displacement estimate.

### Claim 14 (depends on 11)

The electronic device of claim 11, wherein the processor is configured to:
obtain an error metric indicating an uncertainty in the GPS velocity; and
weight the GPS velocity in a Kalman filter based on the error metric.

### Claim 15 (depends on 11)

The electronic device of claim 11, wherein the processor is configured to bound a predicted velocity error based on the GPS velocity.

### Claim 16 (depends on 11)

The electronic device of claim 11, wherein the processor is configured to use the GPS velocity to reduce visual-inertial odometry (VIO) drift.

### Claim 17 (depends on 11)

The electronic device of claim 11, wherein the processor is configured to reject at least one of a set of GPS velocities based on at least one of a set of difference vectors.

### Claim 18 (depends on 11)

The electronic device of claim 11, wherein the processor is configured to reject at least one of a set of GPS velocities based on an error metric.

### Claim 19 (depends on 11)

The electronic device of claim 11, wherein the pose information comprises a 6 degrees of freedom (DOF) camera pose.

### Claim 20 (depends on 11)

The electronic device of claim 11, wherein a vehicle comprises the electronic device.

### Claim 21

A non-transitory tangible computer-readable medium storing computer executable code, comprising:
code for causing an electronic device to determine a predicted velocity relative to Earth corresponding to a first epoch using a camera and an inertial measurement unit (IMU);
code for causing the electronic device to determine, using a Global Positioning System (GPS) receiver, a GPS velocity relative to Earth;
code for causing the electronic device to determine a difference vector between the predicted velocity and the GPS velocity;
code for causing the electronic device to refine a bias estimate and a scale factor estimate of IMU measurements proportional to the difference vector;
code for causing the electronic device to refine a misalignment estimate between the camera and the IMU based on the difference vector; and
code for causing the electronic device to provide pose information based on the refined bias estimate, the refined scale factor estimate, and the refined misalignment estimate.

### Claim 22 (depends on 21)

The computer-readable medium of claim 21, further comprising code for causing the electronic device to determine a second predicted velocity corresponding to a second epoch using the camera and the IMU based on the refined bias estimate, the refined scale factor estimate, and the refined misalignment estimate.

### Claim 23 (depends on 21)

The computer-readable medium of claim 21, further comprising:
code for causing the electronic device to obtain an error metric indicating an uncertainty in the GPS velocity; and
code for causing the electronic device to weight the GPS velocity in a Kalman filter based on the error metric.

### Claim 24 (depends on 21)

The computer-readable medium of claim 21, further comprising code for causing the electronic device to reject at least one of a set of GPS velocities based on at least one of a set of difference vectors.

### Claim 25 (depends on 21)

The computer-readable medium of claim 21, further comprising code for causing the electronic device to reject at least one of a set of GPS velocities based on an error metric.

### Claim 26

An apparatus, comprising:
means for determining a predicted velocity relative to Earth corresponding to a first epoch using a camera and an inertial measurement unit (IMU);
means for determining a GPS velocity relative to Earth;
means for determining a difference vector between the predicted velocity and the GPS velocity;
means for refining a bias estimate and a scale factor estimate of IMU measurements proportional to the difference vector;
means for refining a misalignment estimate between the camera and the IMU based on the difference vector; and
means for providing pose information based on the refined bias estimate, the refined scale factor estimate, and the refined misalignment estimate.

### Claim 27 (depends on 26)

The apparatus of claim 26, further comprising means for determining a second predicted velocity corresponding to a second epoch using the camera and the IMU based on the refined bias estimate, the refined scale factor estimate, and the refined misalignment estimate.

### Claim 28 (depends on 26)

The apparatus of claim 26, further comprising:
means for obtaining an error metric indicating an uncertainty in the GPS velocity; and
means for weighting the GPS velocity in a Kalman filter based on the error metric.

### Claim 29 (depends on 26)

The apparatus of claim 26, further comprising means for rejecting at least one of a set of GPS velocities based on at least one of a set of difference vectors.

### Claim 30 (depends on 26)

The apparatus of claim 26, further comprising means for rejecting at least one of a set of GPS velocities based on an error metric.