# US 2020/0349368 A1 · Enhanced Navigation Instructions with Landmarks Under Difficult Driving Conditions

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0349368 A1
- Document ID: US20200349368A1
- Status: Published application
- Kind: A1
- Application number: 16/617,902
- Filed: 2019-01-09
- Published: 2020-11-05
- Earliest claimed: 2018-11-20
- Assignees: Google Inc.
- Applicants: Google LLC
- Inventors: Yan Mayster; Brian Edmond Brewington
- Web page: https://patentgrep.com/patent/US20200349368A1

## CPC

- G06V 20/582: {of traffic signs}
- G01C 21/3602: {Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera}
- G01C 21/3644: {Landmark guidance, e.g. using POIs or conspicuous other objects}
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S)

## Abstract

The technology relates to factors regarding the external environment around a vehicle that are used to trigger enhanced driving directions for use by the driver. The factors can include issues such as adverse weather conditions, low light conditions and temporary obstacles or other obstructions that may prevent or reduce the ability of the driver to see street signs or other landmarks that are part of an initial set of driving directions. Upon determination of one or more relevant factors, the system is able to modify or otherwise enhances directions in real time. This allows the driver to be able to quickly and easily identify other visible landmarks and use such information to navigate to a desired destination. This approach employs one or more on-board cameras configured to detect objects and conditions in the environment adjacent to or otherwise nearby the vehicle, such as within line of sight of the vehicle's front end.

## Claims

### Claim 1

A method comprising:
obtaining initial directions for a driving route for a driver of a vehicle, the directions for use in routing the vehicle from a starting point to a destination;
obtaining previously captured imagery of one or more locations along the route;
identifying a first landmark located in the previously captured imagery;
receiving, by one or more processors of a computer device, real-time imagery obtained by a camera located in the vehicle;
comparing, by the one or more processors, at least one image from the real-time imagery against at least one image from the previously captured imagery to determine whether the first landmark would be observable by the driver of the vehicle; and
modifying by the one or more processors, upon a determination that the first landmark would not be observable by the driver of the vehicle, the initial directions for the driving route to indicate at least one other landmark different than the first landmark.

### Claim 2 (depends on 1)

The method of claim 1, wherein the comparing to determine whether the first landmark would be observable by the driver of the vehicle includes determining whether the first landmark is partially or fully obscured from viewing by the driver of the vehicle.

### Claim 3 (depends on 2)

The method of claim 2, wherein determining whether the first landmark is partially or fully obscured includes determining that visibility of the first landmark is adversely impacted by a weather condition, a low light condition, another vehicle, a building or vegetation.

### Claim 4 (depends on 1)

The method of any of claim 1, wherein the real-time imagery is obtained by the camera during driving along the route.

### Claim 5 (depends on 1)

The method of claim 1, wherein the modified directions indicate to the driver to take a specific driving action when the at least one other landmark is visible to the driver.

### Claim 6 (depends on 5)

The method of claim 5, wherein the specific driving action is selected from a group consisting of slowing down, turning, stopping or parking.

### Claim 7 (depends on 1)

The method of claim 1, wherein modifying the initial directions includes at least one of updating a graphical display within the vehicle or playing audible instructions within the vehicle.

### Claim 8 (depends on 1)

The method of claim 1, wherein the camera located in the vehicle has a field of view, and comparing the at least one image from the real-time imagery against the at least one image from the previously captured image includes evaluating the camera's field of view against an expected field of view of the driver.

### Claim 9 (depends on 1)

The method of claim 1, wherein modifying the initial directions further includes augmenting or replacing the initial directions with at least one visual clue based on one or more objects along the driving route visible to the camera.

### Claim 10

A method comprising:
receiving, by one or more processors of a computer device, real-time imagery obtained by a camera located in a vehicle during driving of the vehicle by a driver;
analyzing, by the one or more processors, at least one image from the real-time imagery to determine a current condition in an external environment around the vehicle;
determining by the one or more processors, based on the analyzing, that the current condition is expected to obscure a selected landmark in the external environment from viewing by the driver;
upon determining that the current condition is expected to obscure the selected landmark in the external environment from viewing by the driver, identifying by the one or more processors a different landmark that is observable by the camera located in the vehicle; and
providing a visual, audible or haptic indication of the different landmark to the driver.

### Claim 11 (depends on 10)

The method of claim 10, further comprising prior to the providing, modifying an initial set of directions for driving to a destination with the indication of the different landmark.

### Claim 12 (depends on 11)

The method of claim 11, wherein the modified initial directions indicate to the driver to take a specific driving action when the different landmark is visible to the driver.

### Claim 13 (depends on 12)

The method of claim 12, wherein the specific driving action is selected from a group consisting of slowing down, turning, stopping or parking.

### Claim 14 (depends on 10)

The method of claim 10, wherein determining that the current condition is expected to obscure the selected landmark in the external environment from viewing by the driver includes determining that visibility of the selected landmark is adversely impacted by a weather condition, a low light condition, another vehicle, a building or vegetation.

### Claim 15 (depends on 10)

The method of claim 10, wherein analyzing the at least one image from the real-time imagery to determine the current condition in the external environment around the vehicle comprises applying a machine learning model to the real-time imagery in order to detect an obstructions or other visibility impairment in the external environment.

### Claim 16

A system comprising:
memory configured to store at least one of instructions or data; and
one or more processors operatively coupled to the memory, the one or more processors being configured to:
obtain initial directions for a driving route for a driver of a vehicle, the directions for use in routing the vehicle from a starting point to a destination;
obtain previously captured imagery of one or more locations along the route;
identify a first landmark located in the previously captured imagery;
receive real-time imagery obtained by a camera located in the vehicle;
compare at least one image from the real-time imagery against at least one image from the previously captured imagery to determine whether the first landmark would be observable by the driver of the vehicle; and
modify, upon a determination that the first landmark would not be observable by the driver of the vehicle, the initial directions for the driving route to indicate at least one other landmark different than the first landmark.

### Claim 17 (depends on 16)

The system of claim 16, further comprising the camera.

### Claim 18 (depends on 16)

The system of claim 16, wherein the camera is part of a personal communication device.

### Claim 19

A system comprising:
memory configured to store at least one of instructions or data; and
one or more processors operatively coupled to the memory, the one or more processors being configured to:
receive real-time imagery obtained by a camera located in a vehicle during driving of the vehicle by a driver;
analyze at least one image from the real-time imagery to determine a current condition in an external environment around the vehicle;
determine, based on the analysis, that the current condition is expected to obscure a selected landmark in the external environment from viewing by the driver;
upon determining that the current condition is expected to obscure the selected landmark in the external environment from viewing by the driver, identify a different landmark that is observable by the camera located in the vehicle; and
provide a visual, audible or haptic indication of the different landmark to the driver.

### Claim 20 (depends on 19)

The system of claim 19, wherein the analysis of the at least one image from the real-time imagery to determine the current condition in an external environment around the vehicle is performed by the one or more processors according to a machine learning process.

### Claim 21 (depends on 1)

A computing system comprising:
one or more processors; and
a non-transitory computer-readable memory comprising instructions that, when executed, cause the at least one processor to execute the method of any of claim 1.

### Claim 22 (depends on 1)

A non-transitory computer-readable storage medium storing instructions that, when executed, cause one or more processors to carry out the method of any one of claim 1.
