# US 2021/0239485 A1 · System and method for vehicle navigation using terrain text recognition

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0239485 A1
- Document ID: US20210239485A1
- Status: Published application
- Kind: A1
- Application number: 16/782,683
- Filed: 2020-02-05
- Published: 2021-08-05
- Earliest claimed: 2020-02-05
- Assignees: GM Global Technology Operations LLC
- Applicants: GM Global Technology Operations LLC
- Inventors: Farui Peng; Kai-Han Chang; Wei Tong
- Web page: https://patentgrep.com/patent/US20210239485A1

## CPC

- G01C 21/3602: {Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera}
- G01C 21/365: {Guidance using head up displays or projectors, e.g. virtual vehicles or arrows projected on the windscreen or on the road itself}
- G05D 1/0274: {using mapping information stored in a memory device (navigation using map-matching G01C21/30)}
- G05D 1/0278: {using satellite positioning signals, e.g. GPS}
- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06F 18/2148: {characterised by the process organisation or structure, e.g. boosting cascade}
- G06V 20/20: in augmented reality scenes
- G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle
- G08G 1/09623: {Systems involving the acquisition of information from passive traffic signs by means mounted on the vehicle (G08G1/0967 takes precedence)}
- G08G 1/09626: {where the origin of the information is within the own vehicle, e.g. a local storage device, digital map}
- G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S)
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93})

## Abstract

A method of vehicle navigation using terrain text recognition includes receiving, via an electronic controller arranged on a vehicle and having access to a map of the terrain, a navigation route through the terrain. The method also includes receiving, via the controller, a signal from a global positioning system (GPS) to determine a current position of the vehicle relative to the terrain. The method additionally includes determining, via the controller, a location of a next waypoint on the navigation route and relative to the current vehicle position. The method also includes detecting and communicating to the controller, via a vehicle sensor an image frame displaying a text indicative of the next waypoint and correlating, via the controller, the detected text to the next waypoint on the map. Furthermore, the method includes setting, via the controller, an in-vehicle alert indicative of the detected text having been correlated to the next waypoint.

## Claims

### Claim 1

A method of vehicle navigation using terrain text recognition, the method comprising:
receiving, via an electronic controller arranged on a vehicle and having access to a map of the terrain, a navigation route through the terrain;
receiving, via the electronic controller, a signal from a global positioning system (GPS) and using the signal to determine a current position of the vehicle relative to the terrain;
determining, via the electronic controller, a location of a next waypoint on the navigation route and relative to the current position of the vehicle;
detecting and communicating to the electronic controller, via a sensor arranged on the vehicle, an image frame displaying a text indicative of the next waypoint;
correlating, via the electronic controller, the detected text to the next waypoint on the map of the terrain; and
setting, via the electronic controller, an in-vehicle alert indicative of the detected text having been correlated to the next waypoint.

### Claim 2 (depends on 1)

The method according to claim 1, further comprising determining, via the electronic controller, a distance from the current position to the determined location of the next waypoint.

### Claim 3 (depends on 2)

The method according to claim 2, further comprising determining, via the electronic controller, whether the distance from the current position to the determined location of the next waypoint is within a threshold distance.

### Claim 4 (depends on 3)

The method according to claim 3, wherein setting the in-vehicle alert is accomplished when the distance from the current position to the determined location of the next waypoint is within the threshold distance.

### Claim 5 (depends on 1)

The method according to claim 1, wherein correlating the detected text to the next waypoint on the map of the terrain includes using a trained Neural Network architecture.

### Claim 6 (depends on 5)

The method according to claim 5, wherein the Neural Network architecture is a unified Neural Network structure configured to recognize the image frame and including:
a fully-convolutional first Neural Network having an image input and at least one layer, and configured to recognize the text; and
a convolutional second Neural Network having a text input and at least one layer; and
wherein an output from the at least one layer of the second Neural Network is merged with the at least one layer of the first Neural Network, and the first and second Neural Networks are trained together to output a mask score.

### Claim 7 (depends on 6)

The method according to claim 6, wherein setting the in-vehicle alert indicative of the detected text having been correlated to the next waypoint on the map of the terrain includes projecting, via a head-up display (HUD), a highlight icon representative of the mask score onto a view of the next waypoint.

### Claim 8 (depends on 7)

The method according to claim 7, further comprising determining a field of vision of an occupant of the vehicle and setting the in-vehicle alert in response to the determined field of vision.

### Claim 9 (depends on 8)

The method according to claim 8, wherein determining the field of vision includes detecting an orientation of the occupant's eyes and setting the in-vehicle alert includes projecting the highlight icon in response to the detected orientation of the vehicle occupant's eyes.

### Claim 10 (depends on 8)

The method according to claim 8, wherein setting the in-vehicle alert includes triggering an audible signal when the next waypoint appears in the determined field of vision.

### Claim 11

A system for vehicle navigation using terrain text recognition, the system comprising:
an electronic controller arranged on a vehicle and having access to a map of a terrain;
a global positioning system (GPS) in communication with the electronic controller; and
a sensor arranged on the vehicle, in communication with the electronic controller, and configured to detect objects of the terrain;
wherein the electronic controller is configured to:
receive a navigation route through the terrain;
receive a signal from the GPS and use the signal to determine a current position of the vehicle relative to the terrain;
determine a location of a next waypoint on the navigation route and relative to the current position of the vehicle;
receive from the sensor an image frame displaying a text indicative of the next waypoint;
correlate the detected text to the next waypoint on the map of the terrain; and
set an in-vehicle alert indicative of the detected text having been correlated to the next waypoint.

### Claim 12 (depends on 11)

The system according to claim 11, wherein the electronic controller is also configured to determine a distance from the current position to the determined location of the next waypoint.

### Claim 13 (depends on 12)

The system according to claim 12, wherein the electronic controller is additionally configured to determine whether the distance from the current position to the determined location of the next waypoint is within a threshold distance.

### Claim 14 (depends on 13)

The system according to claim 13, wherein the electronic controller is configured to correlate the detected text to the next waypoint on the map of the terrain when the distance from the current position to the determined location of the next waypoint is within the threshold distance.

### Claim 15 (depends on 11)

The system according to claim 11, wherein the electronic controller is configured to correlate the detected text to the next waypoint on the map of the terrain via a trained Neural Network architecture.

### Claim 16 (depends on 15)

The system according to claim 15, wherein the Neural Network architecture is a unified Neural Network structure configured to recognize the image frame and including:
a fully-convolutional first Neural Network having an image input and at least one layer, and configured to recognize the text; and
a convolutional second Neural Network having a text input and at least one layer; and
wherein an output from the at least one layer of the second Neural Network is merged with the at least one layer of the first Neural Network, and the first and second Neural Networks are trained together to output a mask score.

### Claim 17 (depends on 16)

The system according to claim 16, further comprising a head-up display (HUD), wherein the electronic controller is configured to set the in-vehicle alert via projecting, via the HUD, a highlight icon representative of the mask score onto a view of the next waypoint.

### Claim 18 (depends on 17)

The system according to claim 17, wherein the electronic controller is additionally configured to determine a field of vision of an occupant of the vehicle and set the in-vehicle alert in response to the determined field of vision.

### Claim 19 (depends on 18)

The system according to claim 18, wherein the field of vision is determined via detection of an orientation of the occupant's eyes and the in-vehicle alert includes projecting the highlight icon in response to the detected orientation of the vehicle occupant's eyes.

### Claim 20 (depends on 18)

The system according to claim 18, wherein the electronic controller is configured to set the in-vehicle alert via triggering an audible signal when the next waypoint appears in the determined field of vision.