# US 2008/0306708 A1 · System and method for orientation and location calibration for image sensors

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2008/0306708 A1
- Document ID: US20080306708A1
- Status: Published application
- Kind: A1
- Application number: 12/132,423
- Filed: 2008-06-03
- Published: 2008-12-11
- Earliest claimed: 2007-06-05
- Assignees: Raydon Corporation
- Inventors: Edward M. GERMAIN, IV; David Page
- Web page: https://patentgrep.com/patent/US20080306708A1

## CPC

- G01B 21/042: {Calibration or calibration artifacts (G01B3/30, G01B9/02072 take precedence)}
- G06T 7/80: Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
- G06T 2207/10016: Video; Image sequence
- G06T 2207/30204: Marker
- G01B: MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A system and method employing position measurement sensors and point sources of light to determine the location and orientation of video cameras in a simulation arena environment. In an embodiment, one or more accelerometers, gyroscopes, and/or magnetometers associated with each video camera may be used to determine the angular orientation of the video camera. The location of a camera is determined by measuring the distance from the camera to at least two known points, where the known points may be point sources of light, other cameras, or a combination thereof. Camera angular orientation information and camera location information may be combined to provide a complete set of data defining the position of each video camera.

## Claims

### Claim 1

In a simulation environment having a coordinate system, the simulation environment including an image sensor, and a plurality of point sources of light each at a respective fixed and identified location in relation to the coordinate system, a method of determining a position of the image sensor, comprising:
(a) determining an angular orientation of the image sensor in relation to the coordinate system;
(b) determining a plurality of respective distances from the image sensor to the plurality of point sources of light; and
(c) calculating a position of the image sensor in relation to the coordinate system based on the angular orientation and the plurality of respective distances.

### Claim 2 (depends on 1)

The method of claim 1, wherein step (a) comprises determining the angular orientation via an angular orientation determining element associated with the image sensor.

### Claim 3 (depends on 2)

The method of claim 2, wherein determining the angular orientation via the angular orientation determining element comprises determining the angular orientation via at least one of an accelerometer, a magnetometer, or a gyroscope associated with the image sensor.

### Claim 4 (depends on 1)

The method of claim 1, wherein step (b) comprises determining respective angles of incidence of light from the plurality of point source of light at the image sensor.

### Claim 5 (depends on 4)

The method of claim 4, wherein step (b) further comprises performing a distance calculation based on:
the respective angles of incidence; and
a distance between at least a pair of point sources of light of the plurality of point sources of light.

### Claim 6 (depends on 5)

The method of claim 5, wherein the step of performing a distance calculation further comprises determining the distance between the at least a pair of point sources of light based on the respective fixed and identified location in relation to the coordinate system of each point source of light of the at least a pair of point sources of light.

### Claim 7 (depends on 1)

The method of claim 1, wherein the image sensor comprises a stereoscopic image sensor; and
step (b) comprises performing a distance calculation based on a stereoscopic imaging of the plurality of point sources of light.

### Claim 8 (depends on 1)

The method of claim 1, wherein step (c) comprises:
(i) determining equations which define a plurality of spheres, wherein each sphere of the plurality of spheres is centered at a coordinate location of a respective point source of light of the plurality of point sources of light, and wherein each sphere has a respective radius equal to a respective distance from the image sensor to the respective point sources of light.

### Claim 9 (depends on 8)

The method of claim 8, wherein step (c) further comprises:
(ii) determining an intersection of the spheres, wherein the intersection comprises at least one of a circle, a set of points, or a point.

### Claim 10 (depends on 9)

The method of claim 9, wherein step (c) further comprises:
(iii) calculating the position based on the point of intersection of the spheres.

### Claim 11 (depends on 9)

The method of claim 9, wherein step (c) further comprises:
(iii) determining an equation of a line from the image sensor to a point source of light of the plurality of point sources.

### Claim 12 (depends on 11)

The method of claim 11, wherein determining the equation of the line from the image sensor to the point source of light comprises determining the equation of the line based on:
the respective fixed and identified location of the point source in relation to the coordinate system;
the angular orientation of the image sensor in relation to the coordinate system and
an angle of incidence of light from the point source of light at the image sensor.

### Claim 13 (depends on 11)

The method of claim 11, wherein step (c) further comprises:
(iv) calculating an intersection of the line with the intersection of the spheres.

### Claim 14

In a simulation environment having a coordinate system, an image sensor location determination system comprising:
an image sensor;
a plurality of point sources of light each at a respective fixed and identified location in relation to the coordinate system;
at least a processor; and
a memory in communication with the at least a processor;
wherein:
the image sensor is configured to determine an orientation of the image sensor in relation to the coordinate system; and
the memory stores a plurality of processing instructions for directing the at least a processor to determine a position of the image sensor in relation to the coordinate system based on:
the orientation of the image sensor; and
a plurality of respective distances from the image sensor to the plurality of point sources of light.

### Claim 15 (depends on 14)

The system of claim 14, wherein the at least a processor comprises at least one of a processor of the image sensor or a processor of a computer of the simulation environment.

### Claim 16 (depends on 14)

The system of claim 14, further comprising an angular orientation determining element associated with the image sensor, the angular orientation determining element configured to determine the orientation of the image sensor in relation to the coordinate system.

### Claim 17 (depends on 16)

The system of claim 16, wherein the angular orientation determining element comprises at least one of an accelerometer, a magnetometer, or a gyroscope associated with the image sensor.

### Claim 18 (depends on 14)

The system of claim 14, wherein the instructions for directing the at least a processor to determine the location of the image sensor comprise instructions to determine the plurality of respective distances from the image sensor to the plurality of point sources of light.

### Claim 19 (depends on 18)

The system of claim 18, wherein the instructions for directing the at least a processor to determine the plurality of respective distances comprise instructions to calculate the plurality of respective distances based on the respective fixed and identified locations in relation to the coordinate system of the plurality of point sources of light.

### Claim 20 (depends on 18)

The system of claim 18, wherein the instructions for directing the at least a processor to determine the plurality of respective distances comprise instructions to calculate the plurality of respective distances based on the orientation of the image sensor in relation to the coordinate system.

### Claim 21 (depends on 18)

The system of claim 18, wherein the instructions for directing the at least a processor to determine the plurality of respective distances comprise instructions to calculate the plurality of respective distances based on a plurality of respective angles of incidence at the image sensor of light from the plurality of point sources.

### Claim 22 (depends on 21)

The system of claim 21, wherein:
the image sensor is configured to detect an incidence of light from the plurality of point sources of light; and
the instructions for directing the at least a processor to determine the plurality of respective distances further comprise instructions to determine the plurality of respective angles of incidence at the image sensor of light from the plurality of point sources based on the detected incidence of light from the plurality of point sources.

### Claim 23 (depends on 21)

The system of claim 21, wherein:
the image sensor further comprises an imaging element; and
the instructions for directing the at least a processor to determine the plurality of respective angles of incidence at the image sensor of light from the plurality of point sources comprise instructions to determine the plurality of respective angles of incidence based on at least one of:
a location on the imaging element of a ray of light; or
an intensity at the imaging element of the ray of light.

### Claim 24 (depends on 18)

The system of claim 18, wherein:
the image sensor further comprises a stereoscopic image sensor; and
the instructions for directing the at least a processor to determine the plurality of respective distances comprise instructions to determine a distance based on a stereoscopic imaging of the plurality of point sources.

### Claim 25 (depends on 14)

The system of claim 14, wherein the instructions for directing the at least a processor to determine the position of the image sensor in relation to the coordinate system further comprise instructions to determine equations which define a plurality of spheres, wherein:
each sphere of the plurality of spheres is centered around a respective point source of light of the plurality of point sources; and
each sphere has a respective radius equal to the respective distances from the image sensor to the respective point sources of light.

### Claim 26 (depends on 25)

The system of claim 25, wherein the instructions for directing the at least a processor to determine the position of the image sensor in relation to the coordinate system further comprise instructions to determine an intersection of the spheres, wherein the intersection comprises at least one of a circle, a set of points, or a point.

### Claim 27 (depends on 26)

The system of claim 26, wherein the instructions for directing the at least a processor to determine the position of the image sensor in relation to the coordinate system further comprise instructions to calculate the position based on the point of intersection of the spheres.

### Claim 28 (depends on 26)

The system of claim 26, wherein the instructions for directing the at least a processor to determine the position of the image sensor in relation to the coordinate system further comprise instructions to determine an equation of a line from the image sensor to a point source of light of the plurality of point sources.

### Claim 29 (depends on 28)

The system of claim 28, wherein the instructions for directing the at least a processor to determine the equation of a line from the image sensor to the point source of light further comprise instructions to determine the equation of the line based on:
the respective fixed and identified location of the point source in relation to the coordinate system;
the angular orientation of the image sensor in relation to the coordinate system and
an angle of incidence of light from the point source of light at the image sensor.

### Claim 30 (depends on 28)

The system of claim 28, wherein the instructions for directing the at least a processor to determine the position of the image sensor in relation to the coordinate system further comprise instructions to determine an intersection of the line with the intersection of the spheres.

### Claim 31

A computer program product comprising a computer usable medium having control logic stored therein for causing the computer to determine a position of an image sensor in relation to a coordinate system of a simulation environment, the control logic comprising:
first computer readable program code means for causing the computer to receive for a plurality of point sources of light of the simulation environment a plurality of respective fixed and identified locations of the point sources in relation to the coordinate system;
second computer readable program code means for causing the computer to determine the angular orientation of the image sensor in relation to the coordinate system;
third computer readable program code means for causing the computer to calculate a plurality of respective distances from the image sensor to the plurality of point sources of light; and
fourth computer readable program code means for causing the computer to calculate the position of the image sensor in relation to the coordinate system based on the angular orientation and the plurality of respective distances.

### Claim 32 (depends on 31)

The computer program product of claim 31, wherein said second computer readable program code means for causing the computer to determine the angular orientation of the image sensor comprises:
computer readable program code means for causing the computer to determine the angular orientation of the image sensor based on an angular orientation data received from an angular orientation measuring element associated with the image sensor.

### Claim 33 (depends on 32)

The computer program product of claim 32, wherein said second computer readable program code means for causing the computer to determine the angular orientation of the image sensor further comprises:
computer readable program code means for causing the computer to receive the angular orientation data from at least one of an accelerometer, a magnetometer, or a gyroscope associated with the image sensor.

### Claim 34 (depends on 31)

The computer program product of claim 31, wherein said third computer readable program code means for causing the computer to calculate the plurality of respective distances from the image sensor to the plurality of point sources of light comprises:
computer readable program code means for causing the computer to calculate the plurality of respective distances based on the plurality of respective fixed and identified locations of the point sources in relation to the coordinate system.

### Claim 35 (depends on 31)

The computer program product of claim 31, wherein said third computer readable program code means for causing the computer to calculate the plurality of respective distances from the image sensor to the plurality of point sources of light comprises:
computer readable program code means for causing the computer to calculate the plurality of respective distances based on the angular orientation of the image sensor in relation to the coordinate system.

### Claim 36 (depends on 31)

The computer program product of claim 31, wherein said third computer readable program code means for causing the computer to calculate the plurality of respective distances from the image sensor to the plurality of point sources of light comprises:
(i) computer readable program code means for causing the computer to calculate the plurality of respective distances based on a plurality of respective angles of incidence at the image sensor of light from the plurality of point sources.

### Claim 37 (depends on 36)

The computer program product of claim 36, wherein said computer readable program code means for causing the computer to calculate the plurality of respective distances based on a plurality of respective angles of incidence at the image sensor of light from the plurality of point sources comprises:
(i)(a) computer readable program code means for causing the computer to receive from the image sensor a plurality of detected incidences of light from the plurality of point sources detected by the image sensor.

### Claim 38 (depends on 37)

The computer program product of claim 37, wherein a detected incidence of light comprises at least one of a location on an imaging element of the image sensor of a ray of light or an intensity at the imaging element of the ray of light; and
wherein said computer readable program code means for causing the computer to calculate the plurality of respective distances based on the plurality of respective angles of incidence at the image sensor of light from the plurality of point sources further comprises:
(i)(b) computer readable program code means for causing the computer to calculate the plurality of respective angles of incidence based on at least one of:
the location on the imaging element of the image sensor of the ray of light; or
the intensity at the imaging element of the ray of light.

### Claim 39 (depends on 31)

The computer program product of claim 31, wherein said third computer readable program code means for causing the computer to calculate the plurality of respective distances from the image sensor to the plurality of point sources of light further comprises:
computer readable program code means for causing the computer to calculate the plurality of respective distances based on a stereoscopic imaging data of the plurality of point sources by an image sensor configured as a stereoscopic image sensor.

### Claim 40 (depends on 31)

The computer program product of claim 31, wherein said fourth computer readable program code means for causing the computer to calculate the position of the image sensor based on the angular orientation and the plurality of respective distances comprises:
(i) computer readable program code means for causing the computer to determine equations which define a plurality of spheres, wherein:
each sphere of the plurality of spheres is centered around a coordinate location of a respective point source of light of the plurality of point sources; and
each sphere has a respective radius equal to the respective distances from the image sensor to the respective point sources of light.

### Claim 41 (depends on 40)

The computer program product of claim 40, wherein said fourth computer readable program code means for causing the computer to calculate the position of the image sensor based on the angular orientation and the plurality of respective distances further comprises:
(ii) computer readable program code means for causing the computer to calculate an intersection of the spheres, wherein the intersection comprises at least one of a circle, a set of points, or a point.

### Claim 42 (depends on 41)

The computer program product of claim 41, wherein said fourth computer readable program code means for causing the computer to calculate the position of the image sensor based on the angular orientation and the plurality of respective distances further comprises:
(iii) computer readable program code means for causing the computer to determine the position of the image sensor in relation to the coordinate system based on the point of intersection of the spheres.

### Claim 43 (depends on 41)

The computer program product of claim 41, wherein said fourth computer readable program code means for causing the computer to calculate the position of the image sensor based on the angular orientation and the plurality of respective distances further comprises:
(iii) computer readable program code means for causing the computer to calculate an equation of a line from the image sensor to a point source of light of the plurality of point sources.

### Claim 44 (depends on 43)

The computer program product of claim 43, wherein said computer readable program code means for causing the computer to calculate the equation of the line comprises:
(iv) computer readable program code means for causing the computer to calculate the equation of the line based on:
the respective fixed and identified location of the point source in relation to the coordinate system;
the angular orientation of the image sensor in relation to the coordinate system and
an angle of incidence of light from the point source of light at the image sensor.

### Claim 45 (depends on 43)

The computer program product of claim 43, wherein said fourth computer readable program code means for causing the computer to calculate the position of the image sensor based on the angular orientation and the plurality of respective distances further comprises:
(iv) computer readable program code means for causing the computer to calculate an intersection of the line with the intersection of the spheres.
