# US 2023/0054256 A1 · Method and System for Locating a Light Source

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0054256 A1
- Document ID: US20230054256A1
- Status: Published application
- Kind: A1
- Application number: 17/792,153
- Filed: 2021-01-27
- Published: 2023-02-23
- Earliest claimed: 2020-01-28
- Assignees: Fnv IP B.V.
- Applicants: Fnv IP B.V.
- Inventors: Arnoud Marc Jongsma; Joachim Ulrich Seibert
- Web page: https://patentgrep.com/patent/US20230054256A1

## CPC

- G01S 5/16: using electromagnetic waves other than radio waves
- G06T 7/74: {involving reference images or patches}
- B64D 47/00: Equipment not otherwise provided for
- B64F 1/20: Arrangement of optical beacons
- G01S 7/4804: {Auxiliary means for detecting or identifying lidar signals or the like, e.g. laser illuminators}
- G06T 7/66: of image moments or centre of gravity
- G06T 15/06: Ray-tracing
- G06T 15/205: {Image-based rendering}
- G08G 5/21: located onboard the aircraft
- G08G 5/54: for approach or landing
- G08G 5/55: for a single aircraft
- G08G 5/74: for monitoring terrain
- H04N 23/698: for achieving an enlarged field of view, e.g. panoramic image capture
- G01S 2205/03: Airborne
- G06T 2207/10016: Video; Image sequence
- G06T 2207/10032: Satellite or aerial image; Remote sensing
- G06T 2207/30241: Trajectory
- H04N 5/77: between a recording apparatus and a television camera
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- B64D: EQUIPMENT FOR FITTING IN OR TO AIRCRAFT; FLIGHT SUITS; PARACHUTES; ARRANGEMENT OR MOUNTING OF POWER PLANTS OR PROPULSION TRANSMISSIONS IN AIRCRAFT
- B64F: GROUND OR AIRCRAFT-CARRIER-DECK INSTALLATIONS SPECIALLY ADAPTED FOR USE IN CONNECTION WITH AIRCRAFT; DESIGNING, MANUFACTURING, ASSEMBLING, CLEANING, MAINTAINING OR REPAIRING AIRCRAFT, NOT OTHERWISE PROVIDED FOR; HANDLING, TRANSPORTING, TESTING OR INSPECTING AIRCRAFT COMPONENTS, NOT OTHERWISE PROVIDED FOR
- G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93})
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A method and system for locating a high-intensity target light source (26) from an elevated observation location (Po), for instance in an aircraft. The target light source is located at/near an earth surface portion (30) and amongst reference light sources (16, 24, 25) arranged along the surface portion. This target light source emits light (28) with a peak radiant intensity that exceeds the intensity of the reference light sources by at least one order of magnitude. The method includes: acquiring, with an image recording device located at the observation location, images of the light and light emitted the reference light sources; comparing the images and a digital ground map (50) that includes representations of the surface portion and of structures (20, 22) associated with the reference light sources, and estimating a location (Pt) of the target light source relative to the reference light sources, based on the comparison.

## Claims

### Claim 1

A method for locating a target light source, wherein the target light source is located at or near a portion of the earth's surface and amongst a plurality of reference light sources that are arranged along the surface portion, and the target light source emits light with a peak radiant intensity that exceeds the peak radiant intensity of the reference light sources by at least one order of magnitude, and wherein the method comprises:
acquiring, with an image recording device located at an observation location (Po) above the surface portion, for instance at or in an airborne aircraft, one or more images of the light and light emitted by a portion of the reference light sources;
comparing, with a processing unit, the one or more images and a digital ground map, DGM, wherein the DGM includes digital representations of the surface portion and of structures associated with the reference light sources, and;
estimating, with the processing unit, a location (Pt) of the target light source relative to the reference light sources, based on the comparison between the one or more images and the DGM.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the surface portion includes roads, and wherein the reference light sources include road lighting arranged along the roads, and/or automotive lighting of vehicles moving on the roads.

### Claim 3 (depends on 1)

The method according to claim 1, wherein the DGM is geo-referenced relative to an earth-fixed coordinate system, wherein the structures associated with the reference light sources are geo-stationary, the method further comprising:
determining geographic coordinates for the estimated location (Pt) of the target light source, based on the comparison between the one or more images and the DGM;
and optionally communicating the geographic coordinates for the estimated location (Pt) to authorities.

### Claim 4 (depends on 1)

The method according to claim 1, wherein comparing the one or more images and the DGM includes:
iteratively generating view projection images from the DGM, each view projection image being representative of a specific location and viewing angle for the image recording device;
for each generated view projection image, comparing the view projection image to at least one of the images, and calculating a degree of correspondence;
for the at least one of the one or more images, identifying a closest projection image that has a highest degree of correspondence with said one image;
deriving viewing transformation parameters for the closest projection image, and
deriving geographic coordinates for the estimated location for the target light source, from the location of the target light source in the image, the viewing transformation parameters, and the geo-referenced DGM.

### Claim 5 (depends on 4)

The method according to claim 4, wherein comparing the one or more images and the DGM further includes, prior to iteratively generating view projection images:
accessing a set of initial view projection images generated from the DGM, the view projection images representative of typical locations and viewing angles for an image recording device on-board the aircraft, based on a predetermined flight trajectory for the aircraft;
comparing each of the set of initial view projection image to at least one of the images, and calculating a set of initial degrees of correspondence;
identifying from the set a closest initial view projection image having a highest initial degree of correspondence with said one image;
obtaining course viewing transformation parameters for the closest projection image, and
using the course viewing transformation parameters as initialization values for the iteratively generating of view projection images from the DGM.

### Claim 6 (depends on 1)

The method according to claim 1, further comprising:
pre-processing the one or more images to exclude from the images intensity information associated with the light from the target light source while retaining intensity information associated with the plurality of reference light sources;
wherein comparing the one or more images and the DGM is based on the intensity information associated with the plurality of reference light sources, and ignores intensity information associated with the light from the target light source.

### Claim 7 (depends on 4)

The method according to claim 4, wherein estimating the location (Pt) of the target light source comprises:
identifying, in the image, peak intensity coordinates associated with the light from the target source;
for said image, deriving an associated optical centre and viewing axis for the image recording device relative to the reference frame of the DGM, based on the viewing transformation parameters derived for the closest projection image;
using optical ray tracing to calculate a point of intersection between, on the one hand, a nominal axis extending through the peak intensity coordinates and parallel with the viewing axis, and on the other hand, the digital representation of the surface portion in the DGM, and
determining geographic coordinates for the point of intersection, and equating the geographic coordinates with the estimated location (Pt) of the target light source.

### Claim 8 (depends on 7)

The method according to claim 7, wherein the DGM further includes three-dimensional digital representations of buildings that are located in or on the portion of the earth surface, and wherein estimating the location (Pt) of the target light source includes:
using optical ray tracing to calculate a point of intersection between, on the one hand, a nominal axis extending through the peak intensity coordinates and parallel with the viewing axis, and on the other hand, the digital representations of the surface portion and the buildings in the DGM, and
in the case that the nominal axis first intersects a digital representation of a target building in the DGM, determining geographic coordinates including height for the point of intersection, the geographic coordinates optionally including an estimate for the chamber and floor number in the target building.

### Claim 9 (depends on 1)

The method according to claim 1, further comprising:
pre-processing a temporal sequence of the images to identify, in the images, approximate image pixel coordinates for the peak intensity associated with the light from the target light source, by determining if the peak intensity of the light exceeds peak intensities of the reference light sources by at least one order of magnitude within at least one of the images, and by determining if the peak light intensity of the light fluctuates in time by at least one order of magnitude throughout the sequence of the images.

### Claim 10

A system for locating a target light source, wherein the target light source is located at or near a portion (30) of the earth's surface and amongst a plurality of reference light sources (16, 24, 25) that are arranged in an predetermined and essentially stationary spatial distribution along the earth surface portion, and the target light source emits light with a peak radiant intensity that exceeds the peak radiant intensity of the reference light sources by at least one order of magnitude, wherein the observation location is above and remote from the earth surface portion, and wherein the system comprises:
an image recording device, configured to acquire images of the target light source and of the directly surrounding part of the plurality of reference light sources;
a memory unit, provided with a digital ground map, DGM, which includes three-dimensional digital representations of the earth surface portion and the spatial distribution of the reference light sources;
a processing unit, configured to receive the images from the image recording device, to receive the DGM from the memory unit, and to compare the images and the DGM and estimate location (Pt) of the target light source at or near the earth surface portion and relative to the reference light sources.

### Claim 11 (depends on 10)

The system according to claim 10, wherein the image recording device has an ultra-wide field of view with an azimuthal coverage of 360° around a central image recording device axis and an elevational coverage ranging from 0° to at least 90° relative to the image recording device axis (A), and wherein the image recording device is mounted on a lower side on or in the aircraft fuselage, with the image recording device axis pointing downwards.

### Claim 12 (depends on 10)

The system according to claim 10, further comprising a wireless communication unit, configured to transmit the estimated location (Pt) to authorities.

### Claim 13 (depends on 10)

An aircraft, comprising a system for locating a target light source from an observation location (Po) according to claim 10.

### Claim 14 (depends on 1)

A computer program product configured to provide instructions to carry out a method according to claim 1, any when loaded on a computer arrangement.

### Claim 15 (depends on 14)

A non-transitory computer readable medium, comprising a computer program product according to claim 14.
