# US 2024/0202968 A1 · A method, software product, device and system for determining a direction at a position

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0202968 A1
- Document ID: US20240202968A1
- Status: Published application
- Kind: A1
- Application number: 18/553,101
- Filed: 2022-03-29
- Published: 2024-06-20
- Earliest claimed: 2021-03-30
- Assignees: Saab AB
- Applicants: Saab AB
- Inventors: Linnea NILSSON; Jimmy JONSSON; Per-Olof PERSSON
- Web page: https://patentgrep.com/patent/US20240202968A1

## CPC

- G06T 7/73: using feature-based methods
- F41G 3/22: for vehicle-borne armament, e.g. on aircraft
- G06V 10/776: Validation; Performance evaluation
- G06V 20/10: Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59)
- G06V 20/13: Satellite images
- G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle
- H04N 23/695: Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
- H04N 23/90: Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
- F41G 3/02: using an independent line of sight
- G06T 2207/10032: Satellite or aerial image; Remote sensing
- G06T 2207/30184: Infrastructure
- G06T 2207/30212: Military
- G06T 2207/30244: Camera pose
- G06T 2207/30252: Vehicle exterior; Vicinity of vehicle
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- F41G: WEAPON SIGHTS; AIMING (optical aspects thereof G02B)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

The present disclosure relates to a method for determining a direction at a position (211), the method (100) comprises obtaining (110) an orthorectified photo (210) of a region of Earth's surface comprising the position (211), and a coordinate system for the orthorectified photo (210); identifying (120) the position (211) in the orthorectified photo (210); identifying (130) a reference object (212) in the orthorectified photo (210); determining (140) a reference direction (215) between the position (211) and the identified reference object (212); capturing (150) at least one digital photo (220; 230) at the position (211) with a calibrated digital camera (331), wherein at least one of the at least one captured digital photo (220) depicts the reference object (222); and determining (190) the direction based on the at least one captured digital photo (220; 230) and the determined reference direction (215), wherein the determined direction is based on at least one pixel coordinate corresponding to the depicted reference object (222) in the at least one captured digital photo (220) depicting the reference object (222).

## Claims

### Claim 1

A method (100) for determining a direction at a position (211), the method (100) comprises
obtaining (110) an orthorectified photo (210) of a region of Earth's surface comprising the position (211), and a corresponding coordinate system for the region in said orthorectified photo (210),
identifying (120) the position (211) in the orthorectified photo (210), and determining a set of coordinates corresponding to the position (211),
identifying (130) a reference object (212) in the orthorectified photo (210), thereby determining a set of coordinates corresponding to the reference object (212),
determining (140) a reference direction (215) between the position (211) and the identified reference object (212) based on the corresponding sets of coordinates,
capturing (150) at least one digital photo (220; 230) at the position (211) with a calibrated digital camera (331), wherein at least one of the at least one captured digital photo (220) depicts the reference object (222), and
determining (190) the direction based on the at least one captured digital photo (220; 230) and the determined reference direction (215), wherein the determined direction is based on at least one pixel coordinate corresponding to the depicted reference object (222) in the at least one captured digital photo (220) depicting the reference object (222).

### Claim 2 (depends on 1)

The method according to claim 1, further comprising selecting (180) a pixel in the at least one digital photo (220; 230) corresponding to the direction to be determined, and wherein determining (190) the direction further comprises utilizing the pixel coordinate of the selected pixel to determine (190) the direction corresponding to said selected pixel.

### Claim 3 (depends on 2)

The method according to claim 2, wherein capturing (150) at least one digital photo (220; 230) comprises obtaining for each captured digital photo (220; 230) a relative camera orientation, and wherein, if the selected pixel is in a digital photo (230) lacking a depiction of the reference object (222), determining (190) the direction corresponding to said selected pixel is further based on said obtained relative camera orientations of said photo and at least one of the at least one captured digital photo (220) depicting the reference object (222).

### Claim 4 (depends on 2)

The method according to claim 2, wherein selecting (180) the pixel comprises selecting a plurality of pixels, and wherein determining (190) the direction comprises determining a central direction based on said plurality of pixels.

### Claim 5 (depends on 1)

The method according to claim 1, wherein determining (190) the direction further comprises determining a virtual protractor based on the at least one captured digital photo (220; 230), wherein the virtual protractor defines angular relationships between pixel coordinates in the at least one captured digital photo (220; 230).

### Claim 6 (depends on 1)

The method according to claim 1, further comprising identifying (170) at least one additional depicted reference object (234) upon capturing (150) the at least one digital photo at the position, wherein identifying (170) the at least one additional reference object (234) utilizes the captured digital photo (220) depicting the reference object (222), the captured digital photo (230) depicting the additional reference object (234), and the orthorectified photo (210).

### Claim 7 (depends on 6)

The method according to claim 6, wherein:
identifying (170) at least one additional depicted reference object (234) further comprising determining for each identified additional depicted reference object (234) a reference direction, and
determining (190) a direction is further based on the at least one additional depicted reference object (234) and corresponding determined reference direction(s), and/or based on the determined reference direction (215).

### Claim 8 (depends on 1)

The method according to claim 1, wherein capturing (150) at least one digital photo (220; 230) further comprises at least obtaining range data corresponding to the reference object (212), and wherein the method (100) further comprises validating (160) the reference object by comparing the reference object (212) depicted in at least one digital photo (220,230) and the orthorectified photo (210) based on the corresponding range data.

### Claim 9 (depends on 1)

A computer program product comprising a non-transitory computer-readable storage medium (412) having thereon a computer program comprising program instructions, the computer program being loadable into a processor (411) and configured to cause the processor (411) to perform the method (100) for determining a direction at a position (211) according to claim 1.

### Claim 10

System (300) for determining a direction at a position, the system (300) comprises
a computer (310),
a memory storage (320), and
a camera arrangement (330) comprising a digital camera (331),
wherein the digital camera (331) is a calibrated digital camera arranged to capture digital photos (220; 230),
wherein the camera arrangement (330) is arranged to determine a relative orientation of the digital camera (331) during capture,
wherein the memory storage (320) comprises at least one orthorectified photo (210) of a region of Earth's surface and corresponding coordinate system for the region in said orthorectified photo (210), and
wherein the computer (310) is arranged to:
obtain an orthorectified photo (210) of the region of Earth's surface and the corresponding coordinate system for the region in said orthorectified photo (210),
determine the position (211) in the orthorectified photo (210), thereby determining a set of coordinates corresponding to the position (211),
identify a reference object (212) in the orthorectified photo (210), thereby determining a set of coordinates corresponding to the identified reference object (212),
determine a reference direction (215) between the position (211) and the identified reference object (212) based on the corresponding sets of coordinates,
capture and obtain at least one digital photo (220; 230) at the position (211) by controlling the digital camera (331), and
upon obtaining at least one captured digital photo (220) depicting the reference object (222), determine the direction based on said captured digital photo (220; 230) and the reference direction (215), wherein determining the direction is based on pixel coordinates of at least one pixel corresponding to the depicted reference object (222) in said captured digital photo (220), and the determined reference direction.

### Claim 11 (depends on 10)

The system according to claim 10, further comprising camera orientation means (332), wherein the camera orientation means (332) are arranged to orient and provide the relative orientation of the digital camera (331), and wherein the computer (310) is arranged to control the orientation means (332).

### Claim 12 (depends on 10)

The system according to claim 10, further comprising a rangefinder (333), wherein the rangefinder (333) is arranged to provide range data indicative of the distance to objects in the field of view of the digital camera during capture of a digital photo (220,230), and the computer (310) is arranged to validate the depicted reference object (222) by comparing the depicted reference object (222) in at least one digital photo (220,230) and the identified reference object (212) in the orthorectified photo (210) based on the corresponding range data.

### Claim 13 (depends on 10)

The system according to claim 10, further comprising down detection means (334) of the digital camera (331), arranged to provide a downward direction, such as a direction towards the centre of mass of Earth, and wherein the computer (310) is arranged to determine the relative orientation of the digital camera (331) based on said provided downward direction, and/or wherein the computer (310) is arranged to determine the direction based on said provided downward direction.

### Claim 14 (depends on 10)

The system according to claim 10, comprising a plurality of digital cameras (310), wherein at least two digital cameras (310) are arranged in a fixed relative orientation in relation to each other.

### Claim 15 (depends on 10)

A vehicle comprising the system (300) for determining a direction according to claim 10.
