# US 10,503,988 B2 · Method and apparatus for providing goal oriented navigational directions

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,503,988 B2
- Document ID: US10503988B2
- Status: Granted
- Kind: B2
- Application number: 15/673,909
- Filed: 2017-08-10
- Published: 2018-02-15
- Granted: 2019-12-10
- Earliest claimed: 2016-08-10
- Assignees: Xevo Inc.
- Applicants: Xevo Inc.
- Inventors: John Palmer Cordell; Robert Victor Welland; Samuel James McKelvie; John Hayes Ludwig
- Web page: https://patentgrep.com/patent/US10503988B2

## CPC

- G06N 20/00: Machine learning
- B60Q 9/008: {for anti-collision purposes}
- B60R 1/04: mounted inside vehicle {(B60R1/025, B60R1/08 take precedence)}
- B60R 1/062: with remote control for adjusting position {(B60R1/0607 takes precedence)}
- G01C 21/3602: {Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera}
- G01C 21/3605: {Destination input or retrieval}
- G01S 19/485: {whereby the further system is an optical system or imaging system}
- G05D 1/00: Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots (drive control systems specially adapted for autonomous road vehicles B60W60/00)
- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/09: Supervised learning
- G06V 10/95: {structured as a network, e.g. client-server architectures}
- G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle
- G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- G06V 20/586: {of parking space}
- G06V 20/588: {Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road}
- G06V 20/59: inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
- G06V 20/597: {Recognising the driver's state or behaviour, e.g. attention or drowsiness}
- G06V 40/171: {Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships}
- G06V 40/19: Sensors therefor
- G08B 21/06: indicating a condition of sleep, e.g. anti-dozing alarms
- G08G 1/0112: {from the vehicle, e.g. floating car data [FCD]}
- G08G 1/0116: {from roadside infrastructure, e.g. beacons}
- G08G 1/0129: {for creating historical data or processing based on historical data}
- G08G 1/04: using optical or ultrasonic detectors
- G08G 1/096811: {where the route is computed offboard}
- G08G 1/096861: {where the immediate route instructions are output to the driver, e.g. arrow signs for next turn}
- G08G 1/096888: {where input information is obtained using learning systems, e.g. history databases}
- G08G 1/0969: having a display in the form of a map
- G08G 1/143: {inside the vehicles}
- G08G 1/166: {for active traffic, e.g. moving vehicles, pedestrians, bikes}
- H04L 67/10: in which an application is distributed across nodes in the network (software deployment G06F8/60; multiprogramming arrangements G06F9/46)
- H04N 7/188: {Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position (signal generation from motion picture films H04N5/253)}
- H04W 4/02: Services making use of location information
- H04W 4/44: for communication between vehicles and infrastructures, e.g. vehicle-to-cloud [V2C] or vehicle-to-home [V2H]
- B60R 2300/105: using multiple cameras
- B60R 2300/8006: for monitoring and displaying scenes of vehicle interior, e.g. for monitoring passengers or cargo
- G05D 1/0246: {using a video camera in combination with image processing means}
- G06N 3/045: Combinations of networks
- G06N 3/08: Learning methods
- G06V 2201/10: Recognition assisted with metadata
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- B60Q: ARRANGEMENT OF SIGNALLING OR LIGHTING DEVICES, THE MOUNTING OR SUPPORTING THEREOF OR CIRCUITS THEREFOR, FOR VEHICLES IN GENERAL
- B60R: VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S)
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G08B: SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93})
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72)

## Abstract

A method or system is able to refine Global Positioning System (“GPS”) information for guiding a vehicle via extracted metadata using a GPS refinement (“GR”) model managed by a virtuous cycle containing sensors, machine learning center (“MLC”), and a cloud based network (“CBN”). The GR system includes a set of outward facing cameras, a vehicle onboard computer (“VOC”), and GR model. The outward facing cameras mounted on a vehicle are capable of collecting external images representing a surrounding environment in which the vehicle operates. The VOC is configured to generate a positional vehicle location with respect to the surrounding environment in accordance with the external images and historical stored data obtained from CBN. The GR model is configured to generate a driving guidance based on combined information between the positional vehicle location and GPS data.

## Claims

### Claim 1

A system for refining Global Positioning System (“GPS”) navigation guidance for a vehicle, comprising:
a plurality of outward facing cameras mounted on the vehicle and configured to capture external images representing a surrounding environment in which the vehicle operates, wherein the surrounding environment includes a road upon which the vehicle is operating;
a vehicle onboard computer (“VOC”), coupled to the plurality of outward facing cameras, configured to:
analyze the external images to determine an operating lane of the road in which the vehicle is operating;
analyze the external images to identify at least one reference object;
obtain a size and location of the at least one reference object from a cloud-based network;
generate a positional vehicle location with respect to the surrounding environment and the road based on the operating lane of the vehicle and the size and location of the reference object; and
a GPS refinement (“GR”) model coupled to the VOC and configured to:
generate refined GPS navigation information from a combination of GPS location data and the positional vehicle location; and
generate a driving guidance based on the refined GPS navigation information.

### Claim 2 (depends on 1)

The system of claim 1, wherein the at least one reference object is at a known location.

### Claim 3 (depends on 1)

The system of claim 1, wherein the at least one reference object includes a road sign and the location of the road sign is a surveyed position.

### Claim 4 (depends on 1)

The system of claim 1, wherein the driving guidance includes an instruction for a driver of the vehicle to change lanes.

### Claim 5 (depends on 1)

The system of claim 1, wherein the positional vehicle location is calculated in response to a computation of two optical rays from two external images captured by two different outward facing cameras.

### Claim 6 (depends on 1)

The system of claim 1, wherein the VOC generates the positional vehicle location by being further configured to:
analyze the external images for a plurality of landmarks;
obtain a size and location of each of the plurality of landmarks;
determine a distance from the vehicle to each of the plurality of landmarks based on the size of each of the plurality of landmarks and one or more image capture parameters of the plurality of outward facing cameras; and
determine the positional vehicle location using triangulation between the location of each of the plurality of landmarks and the distance from the vehicle to each of the landmarks.

### Claim 7 (depends on 1)

The system of claim 1, wherein the GR model generates the refined GPS navigation information by being further configured to:
determine a next guidance instruction associated with a next exit of the road for the vehicle to follow to a destination;
analyze the external images to identify lane information associated with the next exit of the road;
modify the next guidance instruction based on the lane information and the operating lane of the vehicle; and
output the next guidance instruction to an operator of the vehicle to change lanes or remain in the operating lane for the next exit.

### Claim 8 (depends on 1)

The system of claim 1, wherein the GR model identifies the lane information by being further configured to:
analyze the external images to detect at least one lane marking on the road.

### Claim 9 (depends on 1)

The system of claim 1, GR model identifies the lane information by being further configured to:
analyze the external images to detect an overhead exit sign with lane markings.