# US 2019/0380106 A1 · Indoor positioning system and method based on geomagnetic signals in combination with computer vision

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0380106 A1
- Document ID: US20190380106A1
- Status: Published application
- Kind: A1
- Application number: 16/421,540
- Filed: 2019-05-24
- Published: 2019-12-12
- Earliest claimed: 2018-06-07
- Assignees: Osense Technology Co., Ltd.
- Applicants: Osense Technology Co., Ltd.
- Inventors: Hung-Ya Tsai; You-Kwang Wang
- Web page: https://patentgrep.com/patent/US20190380106A1

## CPC

- H04W 64/003: {locating network equipment}
- G01C 21/08: involving use of the magnetic field of the earth
- G01C 21/1654: {with electromagnetic compass}
- G01C 21/1656: {with passive imaging devices, e.g. cameras}
- G01C 21/206: {specially adapted for indoor navigation}
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06T 2207/30244: Camera pose
- H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72)
- G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

This disclosure provides an indoor positioning system and method based on geomagnetic signals in combination with computer vision, including: a computer vision positioning module for acquiring a spatial image of a surrounding environment to create computer vision coordinates; a geomagnetic signal positioning module for detecting trajectories of the geomagnetic signals for creating geomagnetic signal coordinates; a map generating module including a database for storing the created computer vision coordinates and the created geomagnetic signal coordinates, and an inertial measurement unit for detecting a current position and a traveling trajectory, to generate a computer vision map and a geomagnetic data map; and a weighting module for calculating a computer vision weight and a geomagnetic signal weight based on the created computer vision coordinates and the created geomagnetic signal coordinates, and inputting the computer vision weight and the geomagnetic signal weight in a weight equation to estimate a present location.

## Claims

### Claim 1

An indoor positioning system based on geomagnetic signals in combination with computer vision, comprising:
a computer vision positioning module comprising an camera unit for obtaining a spatial image of a surrounding environment to create computer vision coordinates;
a geomagnetic signal positioning module comprising a geomagnetic sensing unit for detecting trajectories of geomagnetic signals for creating geomagnetic signal coordinates;
a map generating module comprising a database for storing the created computer vision coordinates and the created geomagnetic signal coordinates, and an inertial measurement unit for detecting a current position and a traveling trajectory, so as to combine the computer vision coordinates with the geomagnetic signal coordinates to generate a computer vision map and a geomagnetic data map; and
a weighting module for calculating a computer vision weight and a geomagnetic signal weight based on the created computer vision coordinates and the created geomagnetic signal coordinates, and inputting the computer vision weight and the geomagnetic signal weight in a weight equation to estimate a present location;
wherein the weight equation is:
W
1
(
W
1
+
W
2
)
×
(
x
1
,
z
1
)
+
W
2
(
W
1
+
W
2
)
×
(
x
2
,
z
2
)
,
wherein x and y coordinates of the computer vision positioning module are indicated as x1 and z1, respectively, the computer vision weight is indicated as W1, x and y coordinates of the geomagnetic signal positioning module are indicated as x2 and z2, respectively, and the geomagnetic signal weight is indicated as W2.

### Claim 2 (depends on 1)

The indoor positioning system of claim 1, wherein the geomagnetic signal weight is calculated based on a geomagnetic confidence value, the geomagnetic confidence value is calculated by matching geomagnetic fingerprint distributions measured currently with geomagnetic fingerprint distributions created previously, wherein an equation of calculating the geomagnetic confidence value is:
X
1
=
(
Bx
-
Bx
′
)
2
+
(
By
-
By
′
)
2
+
(
Bz
-
Bz
′
)
2
)
(
Bx
)
2
+
(
By
)
2
+
(
Bz
)
2
×
100

%
,
wherein the geomagnetic confidence value is indicated as X1, the geomagnetic fingerprint distributions created previously are indicated as Bx, By and Bz, and the geomagnetic fingerprint distributions measured currently are indicated as Bx′, By′ and Bz′.

### Claim 3 (depends on 2)

The indoor positioning system of claim 2, wherein an equation of calculating the geomagnetic signal weight is:
W
2
=
1
2

π
×
0.5

e
-
(
x
2
)
2
2
×
(
0.5
)
2
,
wherein the geomagnetic signal weight is indicated as W2, the geomagnetic confidence value is indicated as X2, and an exponential function is indicated as e.

### Claim 4 (depends on 1)

The indoor positioning system of claim 1, wherein the computer vision weight is calculated based on a computer vision confidence value, the computer vision confidence value is calculated by matching Hamming distances between descriptors formed by extracting feature points, wherein an equation of calculating the computer vision confidence value is:
X
2
=
(
1
-
e
n
×
8
)
×
100

%
,
wherein the computer vision confidence value is indicated as X2, a size of the descriptors is indicated as n and is in unit of bits, and a matched hamming distance is indicated as e and is in unit of bits.

### Claim 5 (depends on 4)

The indoor positioning system of claim 4, wherein the descriptors include SIFT-like descriptors or binary descriptors.

### Claim 6 (depends on 5)

The indoor positioning system of claim 5, wherein an equation of calculating the computer vision weight is:
W
2
=
1
2

π
×
0.5

e
-
(
x
2
)
2
2
×
(
0.5
)
2
,
wherein the computer vision weight is indicated as W2, the computer vision confidence value is indicated as X2, and the exponential function is indicated as e.

### Claim 7 (depends on 1)

The indoor positioning system of claim 1, wherein the map generating module comprises a Bag of Words (BOW) model learning module for reducing a dimension of image frames to reduce a matching time.

### Claim 8 (depends on 1)

The indoor positioning system of claim 1, wherein the map generating module utilizes Visual-Inertia Odometry (VIO) algorithm to track a moving trajectory to create the computer vision coordinates.

### Claim 9 (depends on 1)

The indoor positioning system of claim 1, wherein the map generating module utilizes particle filter algorithm to match moving trajectories of differential curve of magnetic fields to create the geomagnetic signal coordinates.

### Claim 10

An indoor positioning method based on geomagnetic signals in combination with computer vision, comprising:
capturing a spatial image of a surrounding environment through an camera unit for creating computer vision coordinates;
detecting trajectories of geomagnetic signals through a geomagnetic sensing unit for creating geomagnetic signal coordinates;
storing the created computer vision coordinates and the geomagnetic signal coordinates in a database;
detecting a current position and a traveling trajectory via an inertial measurement unit to combine the computer vision coordinates with the geomagnetic signal coordinates, so as to generate a computer vision map and a geomagnetic data map; and
calculating a computer vision weight and a geomagnetic signal weight based on the created computer vision coordinates and the created geomagnetic signal coordinates, and estimating a present location by inputting the computer vision weight and the geomagnetic signal weight in a weight equation;
wherein the weight equation is:
W
1
(
W
1
+
W
2
)
×
(
x
1
,
z
1
)
+
W
2
(
W
1
+
W
2
)
×
(
x
2
,
z
2
)
,
wherein x and y coordinates of the computer vision coordinates are indicated as x1 and z1, respectively, the computer vision weight is indicated as W1, x and y coordinates of the geomagnetic signal coordinates are indicated as x2 and z2, respectively, and the geomagnetic signal weight is indicated as W2.

### Claim 11 (depends on 10)

The indoor positioning method of claim 10, wherein the geomagnetic signal weight is calculated based on a geomagnetic confidence value, the geomagnetic confidence value is calculated by matching geomagnetic fingerprint distributions measured currently with geomagnetic fingerprint distributions created previously, wherein the equation of calculating the geomagnetic confidence value is:
X
1
=
(
Bx
-
Bx
′
)
2
+
(
By
-
By
′
)
2
+
(
Bz
-
Bz
′
)
2
)
(
Bx
)
2
+
(
By
)
2
+
(
Bz
)
2
×
100

%
,
wherein the geomagnetic confidence value is indicated as X1, the geomagnetic fingerprint distributions created previously are indicated as Bx, By and Bz, and the geomagnetic fingerprint distributions measured currently are indicated as Bx′, By′ and Bz′.

### Claim 12 (depends on 11)

The indoor positioning method of claim 11, wherein an equation of calculating the geomagnetic signal weight is:
W
2
=
1
2

π
×
0.5

e
-
(
x
2
)
2
2
×
(
0.5
)
2
,
wherein the geomagnetic signal weight is indicated as W2, the geomagnetic confidence value is indicated as X2, and an exponential function is indicated as e.

### Claim 13 (depends on 10)

The indoor positioning method of claim 10, wherein the computer vision weight is calculated based on a computer vision confidence value, the computer vision confidence value is calculated by matching Hamming distances between descriptors formed by extracting feature points, wherein an equation of calculating the computer vision confidence value is:
X
2
=
(
1
-
e
n
×
8
)
×
100

%
,
wherein the computer vision confidence value is indicated as X2, a size of the descriptors is indicated as n and is in unit of bits, and a matched hamming distance is indicated as e and is in unit of bits.

### Claim 14 (depends on 13)

The indoor positioning method of claim 13, wherein the descriptors include SIFT-like descriptors or binary descriptors.

### Claim 15 (depends on 13)

The indoor positioning method of claim 13, wherein an equation of calculating the computer vision weight is:
W
2
=
1
2

π
×
0.5

e
-
(
x
2
)
2
2
×
(
0.5
)
2
,
wherein the computer vision weight is indicated as W2, the computer vision confidence value is indicated as X2, and an exponential function is indicated as e.

### Claim 16 (depends on 10)

The indoor positioning method of claim 10, wherein the creating computer vision coordinates comprises a Bag of Words (BOW) model learning module for reducing a dimension of image frames to reduce a matching time.

### Claim 17 (depends on 10)

The indoor positioning method of claim 10, wherein the creating computer vision coordinates utilizes Visual-Inertia Odometry (VIO) algorithm to track a moving trajectory to create the computer vision coordinates.

### Claim 18 (depends on 10)

The indoor positioning method of claim 10, wherein the creating geomagnetic signal coordinates utilizes particle filter algorithm to match moving trajectories of differential curve of magnetic fields to create the geomagnetic signal coordinates.
