# US 2016/0259044 A1 · Three-dimensional positioning method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2016/0259044 A1
- Document ID: US20160259044A1
- Status: Published application
- Kind: A1
- Application number: 15/156,423
- Filed: 2016-05-17
- Published: 2016-09-08
- Earliest claimed: 2013-01-04
- Assignees: National Central University
- Applicants: National Central University
- Inventors: Liang-Chien Chen; Chin-Jung Yang
- Web page: https://patentgrep.com/patent/US20160259044A1

## CPC

- G01S 13/867: {Combination of radar systems with cameras}
- G01C 21/005: {with correlation of navigation data from several sources, e.g. map or contour matching (G01C21/30 takes precedence)}
- G01S 13/9027: {Pattern recognition for feature extraction}
- G06T 7/55: from multiple images
- G06V 20/13: Satellite images
- B64G 1/1028: {using optical means for mapping, surveying or detection, e.g. of intelligence}
- B64G 1/1035: {using radar for mapping, surveying or detection, e.g. of intelligence}
- G06T 2207/10036: Multispectral image; Hyperspectral image
- G06T 2207/10044: Radar image
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- B64G: COSMONAUTICS; VEHICLES OR EQUIPMENT THEREFOR

## Abstract

A three-dimensional positioning system includes establishing a geometric model for optical AND radar sensors, obtaining rational function conversion coefficients, refining the rational function model and positioning three-dimensional coordinates. The system calculates rational polynomial coefficients from a geometric model of optical AND radar sensors, followed by refining a rational function model by determined ground control points and object image space intersection. The system then measures one or more conjugate points on the optical and radar images. Finally, an observation equation is established by the rational function model to solve and display three-dimensional coordinates.

## Claims

### Claim 1

A three-dimensional positioning system comprising:
a communication module configured to receive optical image data of a target area from one or more optical imagers and radar image data of the target area from one or more radar imagers;
a processor in communication with the communication module;
a display in communication with the processor; and
computer readable storage media in communication with the processor and configured to induce the processor to
(A) receive optical image data of the target area from the one or more optical imagers and to generate a plurality of corresponding optical images;
(B) employ direct geo-referencing to establish a first geometric model of the plurality of optical images;
(C) receive radar image data of the target area from the one or more radar imagers to generate a plurality of corresponding radar images;
(D) determine range data from the plurality of radar images and employ the range data and a Doppler equation to establish a second geometric model of the radar images;
(E) back project the plurality of optical images according to virtual ground control points in the first geometric model for the optical images;
(F) calculate optical image coordinates corresponding to the virtual ground control points using collinear conditions;
(G) back project the radar images according to the virtual ground control points in the second geometric model of the radar images;
(H) calculate radar image coordinates corresponding to the virtual ground control points with the range data and the Doppler equation;
(I) calculate rational polynomial coefficients for the optical images and for the radar images to establish an integrated rational function model;
(J) convert the optical and the radar image coordinates to a rational function space and calculate corresponding rational function space coordinates;
(K) obtain affine conversion coefficients from the rational function space coordinates and the optical and the radar image coordinates according to the ground control points;
(L) complete a linear conversion to correct system error;
(M) execute partial compensation via least squares collocation for amendments to eliminate systematic errors;
(N) measure conjugate points after the rational function model is established and refined from the optical images and from the radar images;
(O) place the conjugate points into the rational function model to establish an observation equation of three-dimensional positioning; and
(P) induce the display to display a position of a target within the target area as a three-dimensional spatial coordinate via a least squares method.

### Claim 2 (depends on 1)

The system of claim 1, wherein at step (B), the processor establishes the optical image geometric model using a direct geographic counterpoint method with a mathematical formula of:
{right arrow over (G)}={right arrow over (P)}+S{right arrow over (U)},
Xi=X(ti)+SiuiX
Yi=Y(ti)+SiuiY
Zi=Z(ti)+SiuiZ,
wherein, {right arrow over (G)} is a vector from Earth's centroid to the ground surface; {right arrow over (P)} is a vector from Earth's centroid to a satellite; Xi, Yi, Zi are respectively ground three-dimensional coordinates; X(ti), Y(ti), Z(ti) are satellite orbital positions; uiX, uiY, uiZ are respectively image observation vectors; Si is an amount of scale; and ti is time.

### Claim 3 (depends on 1)

The system of claim 1, wherein in step (D), the second geometric model of the radar images based on the range data and the Doppler equation has the mathematical formula of:
R
⇀
=
G
⇀
-
P
⇀
,


R
⇀

=

G
⇀
-
P
⇀

,

f
d
=
-
2
λ


R
⇀

t
,
wherein {right arrow over (R)} is a vector from a satellite to a ground point; {right arrow over (G)} is a vector from Earth's centroid to the ground point of the vector; and {right arrow over (P)} is a vector from Earth's centroid to the satellite.

### Claim 4 (depends on 1)

The system of claim 1, wherein the rational function model at step (I) is obtained by getting rational polynomial coefficients according to a plurality of virtual ground control points and a least squares method, based on the rational function model with a mathematical formula of:
S
RFM
=
p
a

(
X
,
Y
,
Z
)
p
b

(
X
,
Y
,
Z
)
=
∑
i
=
0
i
=
3

∑
j
=
0
j
=
3

∑
k
=
0
k
=
3

a
ijk

X
i

Y
j

Z
k
∑
i
=
0
i
=
3

∑
j
=
0
j
=
3

∑
k
=
0
k
=
3

b
ijk

X
i

Y
j

Z
k
L
RFM
=
p
c

(
X
,
Y
,
Z
)
p
d

(
X
,
Y
,
Z
)
=
∑
i
=
0
i
=
3

∑
j
=
0
j
=
3

∑
k
=
0
k
=
3

c
ijk

X
i

Y
j

Z
k
∑
i
=
0
i
=
3

∑
j
=
0
j
=
3

∑
k
=
0
k
=
3

d
ijk

X
i

Y
j

Z
k
,
wherein aijk, bijk, cijk and dijk are respectively rational function coefficients.

### Claim 5 (depends on 1)

The system of claim 1, wherein at step (K), the rational function model is refined by correcting the rational function model via affine transformation with a mathematical formula of:
Ŝ=A0×SRFM+A1×LRFM+A2
{circumflex over (L)}=A3×SRFM+A4×LRFM+A5
wherein Ŝ and {circumflex over (L)} are respectively corrected image coordinates and A0˜5 are affine conversion coefficients.

### Claim 6 (depends on 1)

The system of claim 1, wherein at step (O), the observation equation of the three-dimensional positioning has a mathematical formula of:
[
υ
S
1
υ
L
1
υ
S
2
υ
L
2
]
=
[
∂
S
1
∂
X
∂
S
1
∂
Y
∂
S
1
∂
Z
∂
L
1
∂
X
∂
L
1
∂
Y
∂
L
1
∂
Z
∂
S
2
∂
X
∂
S
2
∂
Y
∂
S
2
∂
Z
∂
L
2
∂
X
∂
L
2
∂
Y
∂
L
2
∂
Z
]

[
dX
dY
dZ
]
+
[
S
^
1
-
S
1
L
^
1
-
L
1
S
^
2
-
S
2
L
^
2
-
L
2
]
.

### Claim 7 (depends on 1)

The system of claim 1, wherein in step (C), the plurality of radar images is of synthetic aperture radar images.

### Claim 8 (depends on 1)

The system of claim 1, wherein the one or more optical imagers and the one or more radar imagers each comprise a plurality of different types of imagers.

### Claim 9 (depends on 8)

The system of claim 8, wherein the plurality of radar imagers comprises a ALOS/PALSAR satellite-based imager and a COSMO-SkyMed satellite-based imager and wherein the plurality of optical imagers comprises a ALOS/PRISM optical satellite-based imager, a SPOT-5 panchromatic optical satellite-based imager, and a SPOT-5 Super mode optical satellite-based imager.

### Claim 10

Computer readable storage media configured to induce a processor and associated display to
(A) receive optical image data of a target area from one or more optical imagers and to generate a plurality of corresponding optical images;
(B) employ direct geo-referencing to establish a first geometric model of the plurality of optical images;
(C) receive radar image data of the target area from one or more radar imagers to generate a plurality of corresponding radar images;
(D) determine range data from the plurality of radar images and employ the range data and a Doppler equation to establish a second geometric model of the radar images;
(E) back project the plurality of optical images according to virtual ground control points in the first geometric model for the optical images;
(F) calculate optical image coordinates corresponding to the virtual ground control points using collinear conditions;
(G) back project the radar images according to the virtual ground control points in the second geometric model of the radar images;
(H) calculate radar image coordinates corresponding to the virtual ground control points with the range data and the Doppler equation;
(I) calculate rational polynomial coefficients for the optical images and for the radar images to establish an integrated rational function model;
(J) convert the optical and the radar image coordinates to a rational function space and calculate corresponding rational function space coordinates;
(K) obtain affine conversion coefficients from the rational function space coordinates and the optical and the radar image coordinates according to the ground control points;
(L) complete a linear conversion to correct system error;
(M) execute partial compensation via least squares collocation for amendments to eliminate systematic errors;
(N) measure conjugate points after the rational function model is established and refined from the optical images and from the radar images;
(O) place the conjugate points into the rational function model to establish an observation equation of three-dimensional positioning; and
(P) display a position of a target within the target area as a three-dimensional spatial coordinate via a least squares method.

### Claim 11 (depends on 10)

The computer readable storage media of claim 10, wherein at step (B), the optical image geometric model is established using a direct geographic counterpoint method with a mathematical formula of:
{right arrow over (G)}={right arrow over (P)}+S{right arrow over (U)},
Xi=X(ti)+SiuiX
Yi=Y(ti)+SiuiY
Zi=Z(ti)+SiuiZ,
wherein, {right arrow over (G)} is a vector from Earth's centroid to the ground surface; {right arrow over (P)} is a vector from Earth's centroid to a satellite; Xi, Yi, Zi are respectively ground three-dimensional coordinates; X(ti), Y(ti), Z(ti) are satellite orbital positions; uiX, uiY, uiZ are respectively image observation vectors; Si is an amount of scale; and ti is time.

### Claim 12 (depends on 10)

The computer readable storage media of claim 10, wherein in step (D), the second geometric model of the radar images based on the range data and the Doppler equation has the mathematical formula of:
R
⇀
=
G
⇀
-
P
⇀
,


R
⇀

=

G
⇀
-
P
⇀

,

f
d
=
-
2
λ


R
⇀

t
,
wherein {right arrow over (R)} is a vector from a satellite to a ground point; {right arrow over (G)} is a vector from Earth's centroid to the ground point of the vector; and {right arrow over (P)} is a vector from Earth's centroid to the satellite.

### Claim 13 (depends on 10)

The computer readable storage media of claim 10, wherein the rational function model at step (I) is obtained by getting rational polynomial coefficients according to a plurality of virtual ground control points and a least squares method, based on the rational function model with a mathematical formula of:
S
RFM
=
p
a

(
X
,
Y
,
Z
)
p
b

(
X
,
Y
,
Z
)
=
∑
i
=
0
i
=
3

∑
j
=
0
j
=
3

∑
k
=
0
k
=
3

a
ijk

X
i

Y
j

Z
k
∑
i
=
0
i
=
3

∑
j
=
0
j
=
3

∑
k
=
0
k
=
3

b
ijk

X
i

Y
j

Z
k
L
RFM
=
p
c

(
X
,
Y
,
Z
)
p
d

(
X
,
Y
,
Z
)
=
∑
i
=
0
i
=
3

∑
j
=
0
j
=
3

∑
k
=
0
k
=
3

c
ijk

X
i

Y
j

Z
k
∑
i
=
0
i
=
3

∑
j
=
0
j
=
3

∑
k
=
0
k
=
3

d
ijk

X
i

Y
j

Z
k
,
wherein aijk, bijk, cijk and dijk are respectively rational function coefficients.

### Claim 14 (depends on 10)

The computer readable storage media of claim 10, wherein at step (K), the rational function model is refined by correcting the rational function model via affine transformation with a mathematical formula of:
Ŝ=A0×SRFM+A1×LRFM+A2
{circumflex over (L)}=A3×SRFM+A4×LRFM+A5
wherein Ŝ and {circumflex over (L)} are respectively corrected image coordinates and A0˜5 are affine conversion coefficients.

### Claim 15 (depends on 10)

The computer readable storage media of claim 10, wherein at step (O), the observation equation of the three-dimensional positioning has a mathematical formula of:
[
υ
S
1
υ
L
1
υ
S
2
υ
L
2
]
=
[
∂
S
1
∂
X
∂
S
1
∂
Y
∂
S
1
∂
Z
∂
L
1
∂
X
∂
L
1
∂
Y
∂
L
1
∂
Z
∂
S
2
∂
X
∂
S
2
∂
Y
∂
S
2
∂
Z
∂
L
2
∂
X
∂
L
2
∂
Y
∂
L
2
∂
Z
]

[
dX
dY
dZ
]
+
[
S
^
1
-
S
1
L
^
1
-
L
1
S
^
2
-
S
2
L
^
2
-
L
2
]
.

### Claim 16 (depends on 10)

The computer readable storage media of claim 10, wherein in step (C), the plurality of radar images is of synthetic aperture radar images.

### Claim 17 (depends on 10)

The computer readable storage media of claim 10, wherein the one or more optical imagers and the one or more radar imagers each comprise a plurality of different types of imagers.

### Claim 18 (depends on 17)

The system of claim 17, wherein the plurality of radar imagers comprises a ALOS/PALSAR satellite-based imager and a COSMO-SkyMed satellite-based imager and wherein the plurality of optical imagers comprises a ALOS/PRISM optical satellite-based imager, a SPOT-5 panchromatic optical satellite-based imager, and a SPOT-5 Super mode optical satellite-based imager.
