# US 2017/0322043 A1 · Vision augmented navigation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2017/0322043 A1
- Document ID: US20170322043A1
- Status: Published application
- Kind: A1
- Application number: 15/596,790
- Filed: 2017-05-16
- Published: 2017-11-09
- Earliest claimed: 2013-06-13
- Assignees: Mobileye Vision Technologies Ltd.
- Applicants: Mobileye Vision Technologies Ltd.
- Inventors: GIDEON STEIN
- Web page: https://patentgrep.com/patent/US20170322043A1

## CPC

- G01C 21/3602: {Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera}
- G01C 21/3644: {Landmark guidance, e.g. using POIs or conspicuous other objects}
- G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S)

## Abstract

A navigation system may include at least one processing device configured to determine, based on an output of one or more position sensors associated with the navigation system, a current location of at least one component associated with the navigation system and determine a destination location different from the current location. The navigation system may also acquire, from one or more image acquisition devices, a plurality of images representative of an environment of a user of the navigation system and derive, from the plurality of images, visual information associated with at least one object in the environment. The system may also determine one or more instructions for navigating from the current location to the destination location, wherein the one or more instructions include at least one reference to the visual information derived from the plurality of images. The system may also deliver to the user the one or more instructions.

## Claims

### Claim at position 1

1-29. (canceled)

### Claim 30

A navigation system, comprising:
at least one processing device configured to:
determine, based on an output of one or more position sensors associated with the navigation system, a current location of at least one component associated with the navigation system;
determine a destination location different from the current location;
acquire, from one or more image acquisition devices, while a user of the navigation system is located in an environment and navigating from the current location to the destination location, a plurality of images representative of the environment;
analyze, while the user is navigating from the current location to the destination location, the plurality of images to identify at least one object appearing within a field of view of at least one of the image acquisition devices;
determine, based on the plurality of images, a distance from at least one of the one or more image acquisition devices to the at least one object;
determine, based on the determined distance, one or more instructions for navigating from the current location to the destination location, wherein the one or more instructions include a description of the determined distance; and
deliver to the user the one or more instructions.

### Claim 31 (depends on 30)

The navigation system of claim 30, wherein the at least one object includes at least one of a vehicle, a building, a sign, a traffic light, a road feature, a structure, or a person.

### Claim 32 (depends on 30)

The navigation system of claim 30, wherein determining the distance from at least one of the one or more image acquisition devices to the at least one object includes calculating a depth map using structure from motion, and wherein the plurality of images includes at least one image captured at a different time from another of the plurality of images.

### Claim 33 (depends on 32)

The navigation system of claim 32, wherein calculating the depth map further includes using information obtained from at least one of a speedometer or the one or more position sensors.

### Claim 34 (depends on 32)

The navigation system of claim 32, wherein calculating the depth map further includes receiving at least a portion of the depth map and updating the received depth map using the plurality of images.

### Claim 35 (depends on 30)

The navigation system of claim 30, wherein determining the distance from at least one of the one or more image acquisition devices to the at least one object includes:
correlating a first area of a first image in the plurality of images with a second area of a second image in the plurality of images;
warping at least one of the first image and the second image using the correlation;
calculating a disparity map between the first image and the second image; and
determining the distance using the disparity map.

### Claim 36 (depends on 35)

The navigation system of claim 35, wherein determining the distance using the disparity map further includes selecting a portion of the disparity map having a disparity within a range of an expected value, wherein the expected value correlates to a target distance.

### Claim 37 (depends on 30)

The navigation system of claim 30, wherein the one or more instructions are delivered audibly to the user.

### Claim 38

A vehicle, comprising:
one or more position sensors configured to provide an output representative of a position of the vehicle;
one or more image acquisition devices configured to acquire a plurality of images of an environment outside of the vehicle; and
at least one processing device configured to:
determine, based on an output of the one or more position sensors, a current location of at least one component associated with the vehicle;
determine a destination location different from the current location;
acquire, from the one or more image acquisition devices, while a user of the vehicle is located in an environment and navigating from the current location to the destination location, a plurality of images representative of the environment;
analyze, while the user is navigating from the current location to the destination location, the plurality of images to identify at least one object appearing within a field of view of at least one of the image acquisition devices;
determine, based on the plurality of images, a distance from at least one of the one or more image acquisition devices to the at least one object;
determine, based on the determined distance, one or more instructions for navigating from the current location to the destination location, wherein the one or more instructions include a description of the determined distance; and
deliver to the user the one or more instructions.

### Claim 39 (depends on 38)

The vehicle of claim 38, wherein the at least one object includes at least one of a vehicle, a building, a sign, a traffic light, a road feature, a structure, or a person.

### Claim 40 (depends on 38)

The vehicle of claim 38, wherein determining the distance from at least one of the one or more image acquisition devices to the at least one object includes calculating a depth map using structure from motion, and wherein the plurality of images includes at least one image captured at a different time from another of the plurality of images.

### Claim 41 (depends on 40)

The vehicle of claim 40, further comprising:
a speedometer,
wherein the one or more position sensors include a GPS unit, and
wherein calculating the depth map further includes using information obtained from the speedometer or the GPS unit.

### Claim 42 (depends on 40)

The vehicle of claim 40, wherein calculating the depth map further includes receiving at least a portion of the depth map and updating the received depth map using the plurality of images.

### Claim 43 (depends on 38)

The vehicle of claim 38, wherein determining the distance from at least one of the one or more image acquisition devices to the at least one object includes:
correlating a first area of a first image in the plurality of images with a second area of a second image in the plurality of images;
warping at least one of the first image and the second image using the correlation;
calculating a disparity map between the first image and the second image; and
determining the distance using the disparity map.

### Claim 44 (depends on 43)

The vehicle of claim 43, wherein determining the distance using the disparity map further includes selecting a portion of the disparity map having a disparity within a range of an expected value, wherein the expected value correlates to a target distance.

### Claim 45 (depends on 38)

The vehicle of claim 38, further comprising:
at least one speaker, and
wherein the one or more instructions are delivered audibly to the user using the at least one speaker.

### Claim 46

A method for navigating a vehicle, comprising:
determining, based on an output of one or more position sensors associated with the navigation system, a current location of at least one component associated with the navigation system;
determining a destination location different from the current location;
acquiring, from one or more image acquisition devices, while a user of the navigation system is located in an environment and navigating from the current location to the destination location, a plurality of images representative of the environment;
analyzing, while the user is navigating from the current location to the destination location, the plurality of images to identify at least one object appearing within a field of view of at least one of the image acquisition devices;
determining, based on the plurality of images, a distance from at least one of the one or more image acquisition devices to the at least one object;
determining, based on the determined distance, one or more instructions for navigating from the current location to the destination location, wherein the one or more instructions include a description of the determined distance; and
deliver to the user the one or more instructions.

### Claim 47 (depends on 46)

The method of claim 46, wherein determining the distance from at least one of the one or more image acquisition devices to the at least one object includes calculating a depth map using structure from motion, and wherein the plurality of images includes at least one image captured at a different time from another image.

### Claim 48 (depends on 47)

The method of claim 47, wherein calculating the depth map further includes using information obtained from at least one of a speedometer or the one or more position sensors.

### Claim 49 (depends on 46)

The method of claim 46, wherein determining the distance from at least one of the one or more image acquisition devices to the at least one object includes:
correlating a first area of a first image in the plurality of images with a second area of a second image in the plurality of images;
warping at least one of the first image and the second image using the correlation;
calculating a disparity map between the first image and the second image; and
determining the distance using the disparity.
