# US 2020/0401617 A1 · Visual positioning system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0401617 A1
- Document ID: US20200401617A1
- Status: Published application
- Kind: A1
- Application number: 16/703,705
- Filed: 2019-12-04
- Published: 2020-12-24
- Earliest claimed: 2019-06-21
- Assignees: White Raven Ltd.
- Applicants: White Raven Ltd
- Inventors: Ehud Spiegel; Shai Peer; Boaz Shvartzman; Aaron Demri; Ofer Avni
- Web page: https://patentgrep.com/patent/US20200401617A1

## CPC

- G06F 16/5866: {using information manually generated, e.g. tags, keywords, comments, manually generated location and time information}
- G06F 16/532: Query formulation, e.g. graphical querying
- G06F 16/587: using geographical or spatial information, e.g. location
- G06F 18/211: Selection of the most significant subset of features
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06V 10/454: {Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]}
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/82: using neural networks
- G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle
- G06N 3/045: Combinations of networks
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Systems and methods for image-based self-localization of an image capture device, include receiving a query image of an undetermined location of the image and comparing the query image to a set of geotagged reference images to determine a location of the query image. The system and method may alternatively compare a series of query images to reference images or portions of feature vectors generated from said query images and reference images to determine the closest query image in the series to respective reference images. The set of geotagged reference images may include a sequence of previously obtained reference images of a route, each reference image corresponding to a known geolocation. The geolocation of the user may be determined based on the location of the query image within the set. Ambient conditions of the images may be used to improve comparison of a query image to reference images. Segment and/or abstractions of cell images may be used to reduce computational and/or communications resources.

## Claims

### Claim 1

A method for image based self-localization of a user comprising the steps of:
receiving one or more query images at a processor;
identifying an approximate location of query image capture;
selecting a subset of geotagged reference images proximal to said approximate location of query image capture, wherein said reference images, each correspond to a known geolocation;
using the processor to compare said query images to a said subset of geotagged reference images to determine which reference image most closely matches a query image; and
using the processor to determine a geolocation of the capture of said query image by reference to the geotag of the reference image most similar to said query image.

### Claim 2 (depends on 1)

The method of claim 1 comprising using the processor to control a device based on determination of the geolocation of the user.

### Claim 3 (depends on 1)

The method of claim 1 comprising:
assigning a location sensitive tag to the query image; and
comparing the tag of the query image to tags linked to each reference image to determine the location capture of the query image.

### Claim 4 (depends on 1)

The method of claim 1 comprising using the processor to determine the location of capture of the query image within the reference images based on a known parameter of the user's movement.

### Claim 5 (depends on 1)

The method of claim 1 wherein the step of comparing comprises using a Convolutional Neural Network.

### Claim 6 (depends on 1)

The method of claim 1 wherein the reference images are linked to known ambient conditions.

### Claim 7 (depends on 1)

The method of claim 1 comprising using the query image to update the reference images.

### Claim 8 (depends on 1)

The method of claim 1 comprising using the processor to select a subset of reference images based on a previously determined geolocation of the image capture and based on a probable route.

### Claim 9 (depends on 1)

The method of claim 1 comprising:
assigning a location sensitive tag to the query image;
comparing the tag assigned to the query image to tags linked to each reference image in the sequence of reference images, the tags sensitive to location of each reference image within the sequence, to determine a location of the query image within the sequence; and
determining the matching image based on the location of the query image within the sequence.

### Claim 10 (depends on 9)

The method of claim 9 wherein determining ambient conditions is based on information automatically obtained from a time recording device associated with the user.

### Claim 11 (depends on 10)

The method of claim 10 comprising:
determining a location of the first matching reference image within a sequence of the first set of reference images to determine a first geolocation;
determining a location of the second matching reference image within a sequence of the second set of reference images to determine a second geolocation;
combining the first and second geolocations to a combined geolocation; and
determining the geolocation of the user based on the combined geolocation.

### Claim 12 (depends on 1)

The method of claim 1 comprising using the query image to update the sequence of images linked to known ambient conditions.

### Claim 13 (depends on 1)

The method according to claim 1 wherein one or more reference images are processed to determine features that contrast similar reference images proximal said one or more references image(s);
weighting said features that contrast in the reference images in the processor determination of similarity of said reference images to said query image.

### Claim 14

A method for self-geolocation comprising the steps of:
determining approximate location of a query image capture;
identifying a geotagged reference image captured proximally to said location of said image capture, wherein said geotagged reference image includes a marked point of interest;
comparing a series of query images to said identified geotagged reference image to find the query image that best matches the reference image;
tracking a current location relative to a location of capture of the query image that best matches the reference image.

### Claim 15 (depends on 14)

The method according to claim 14 further comprising the step of identifying aspects of a query image corresponding to a marked point of interest in said reference image to determine the relative location of the query image that best matches the reference image and the geotagged position of the reference image.
