# US 12,110,037 B2 · Navigation and mapping based on detected arrow orientation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,110,037 B2
- Document ID: US12110037B2
- Status: Granted
- Kind: B2
- Application number: 17/096,495
- Filed: 2020-11-12
- Published: 2021-03-04
- Granted: 2024-10-08
- Earliest claimed: 2018-05-15
- Assignees: Mobileye Vision Technologies Ltd.
- Applicants: Mobileye Vision Technologies Ltd.
- Inventors: Avigdor Eldar; Eli Peker; Matan Sulami
- Web page: https://patentgrep.com/patent/US12110037B2

## CPC

- B60W 60/001: {Planning or execution of driving tasks}
- B60T 7/12: for automatic initiation; for initiation not subject to will of driver or passenger {(limiting speed of vehicles other than rail vehicles B60K31/00)}
- B60W 10/18: including control of braking systems
- B60W 10/20: including control of steering systems
- B60W 30/12: Lane keeping
- B60W 30/18072: {Coasting}
- B60W 30/18109: {Braking}
- B62D 6/001: {the torque NOT being among the input parameters}
- G01C 21/30: Map- or contour-matching
- G01C 21/3407: {specially adapted for specific applications}
- G01C 21/3602: {Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera}
- G01C 21/3626: {Details of the output of route guidance instructions (traffic control systems for road vehicles involving transmission of navigation instructions to the vehicle G08G1/0968)}
- G01C 21/3658: {Lane guidance}
- G01C 21/3691: {Retrieval, searching and output of information related to real-time traffic, weather, or environmental conditions (arrangements for giving variable traffic instructions G08G1/09)}
- G01C 21/3804: {Creation or updating of map data}
- G01C 21/3815: {Road data}
- G01C 21/3848: {Data obtained from both position sensors and additional sensors}
- G01C 21/3885: {Transmission of map data to client devices; Reception of map data by client devices}
- G05D 1/0088: {characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours (using knowledge based models G06N5/00)}
- G05D 1/0214: {in accordance with safety or protection criteria, e.g. avoiding hazardous areas (monitoring the location of vehicles within a certain area, e.g. forbidden or allowed areas, in traffic control systems for road vehicles G08G1/13)}
- G05D 1/0221: {involving a learning process}
- G05D 1/0246: {using a video camera in combination with image processing means}
- G05D 1/0278: {using satellite positioning signals, e.g. GPS}
- G05D 1/0287: {involving a plurality of land vehicles, e.g. fleet or convoy travelling (traffic control systems for road vehicles G08G1/00, particularly anticollision systems G08G1/16)}
- G05D 1/248: generated by satellites, e.g. GPS
- G05D 1/249: from positioning sensors located off-board the vehicle, e.g. from cameras
- G05D 1/617: Safety or protection, e.g. defining protection zones around obstacles or avoiding hazards (arrangements for controlling the position or course of two or more vehicles for avoiding collisions therebetween G05D1/693; arrangements for reacting to or preventing system or operator failure G05D1/80)
- G05D 1/692: involving a plurality of disparate vehicles
- G05D 1/81: Handing over between on-board automatic and on-board manual control
- G06N 3/08: Learning methods
- G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle
- G06V 20/584: {of vehicle lights or traffic lights}
- G06V 20/588: {Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road}
- G08G 1/0112: {from the vehicle, e.g. floating car data [FCD]}
- G08G 1/07: Controlling traffic signals
- G08G 1/09623: {Systems involving the acquisition of information from passive traffic signs by means mounted on the vehicle (G08G1/0967 takes precedence)}
- G08G 1/143: {inside the vehicles}
- G08G 1/167: {Driving aids for lane monitoring, lane changing, e.g. blind spot detection}
- G08G 1/22: {Platooning, i.e. convoy of communicating vehicles}
- B60W 2300/123: Light trucks
- B60W 2420/403: Image sensing, e.g. optical camera
- B60W 2520/06: Direction of travel
- B60W 2520/12: Lateral speed
- B60W 2552/05: Type of road, e.g. motorways, local streets, paved or unpaved roads
- B60W 2552/30: Road curve radius
- B60W 2552/45: Pedestrian sidewalk
- B60W 2552/53: Road markings, e.g. lane marker or crosswalk
- B60W 2554/60: Traversable objects, e.g. speed bumps or curbs
- B60W 2555/60: Traffic rules, e.g. speed limits or right of way
- B60W 2556/40: High definition maps
- B60W 2556/55: using telemetry
- B60W 2556/65: Data transmitted between vehicles
- G01C 21/38: {Electronic maps specially adapted for navigation; Updating thereof}
- G01C 21/3807: {characterised by the type of data}
- G01C 21/3841: {Data obtained from two or more sources, e.g. probe vehicles}
- G05D 2101/15: using machine learning, e.g. neural networks
- G07C 5/0808: {Diagnosing performance data (testing of vehicles G01M17/00; testing of electrical installation on vehicles G01R31/005)}
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60T: VEHICLE BRAKE CONTROL SYSTEMS OR PARTS THEREOF; BRAKE CONTROL SYSTEMS OR PARTS THEREOF, IN GENERAL (electrodynamic brake systems for vehicle, in general B60L; brakes per se, i.e. devices where braking effect occurs, including ultimate brake actuators, F16D); ARRANGEMENT OF BRAKING ELEMENTS ON VEHICLES IN GENERAL; PORTABLE DEVICES FOR PREVENTING UNWANTED MOVEMENT OF VEHICLES; VEHICLE MODIFICATIONS TO FACILITATE COOLING OF BRAKES
- B62D: MOTOR VEHICLES; TRAILERS (steering, or guiding on a desired track, of agricultural machines or implements A01B69/00; wheels, castors, axles, increasing wheel adhesion B60B; vehicle tyres, tyre inflation or tyre changing B60C; connections between vehicles of a train or the like B60D; vehicles for use on rail and road, amphibious or convertible vehicles B60F; suspension arrangements B60G; heating, cooling, ventilating or other air treating devices B60H; windows, windscreens, non-fixed roofs, doors or similar devices, protective coverings for vehicles not in use B60J; propulsion plant arrangements, auxiliary drives, transmissions, controls, instrumentation or dashboards B60K; electric equipment or propulsion of electrically-propelled vehicles B60L; power supply for electrically-propelled vehicles B60M; passenger accommodation not otherwise provided for B60N; adaptations for load transportation or to carry special loads or objects B60P; arrangement of signalling or lighting devices, the mounting or supporting thereof or circuits therefor, for vehicles in general B60Q; vehicles, vehicle fittings or vehicle parts not otherwise provided for B60R; servicing, cleaning, repairing, supporting, lifting, or manoeuvring, not otherwise provided for, B60S; brake arrangements, brake control systems or parts thereof B60T; air-cushion vehicles B60V; motor-cycles, accessories therefor B62J, B62K; testing of vehicles G01M)
- G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S)
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93})
- G07C: TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE

## Abstract

A system for mapping a lane mark for use in autonomous vehicle navigation is provided. The system includes at least one processor programmed to: receive two or more location identifiers associated with a detected lane mark; associate the detected lane mark with a corresponding road segment; update an autonomous vehicle road navigation model relative to the corresponding road segment based on the two or more location identifiers associated with the detected lane mark; and distribute the updated autonomous vehicle road navigation model to a plurality of autonomous vehicles.

## Claims

### Claim 1

A system for mapping a directional arrow for use in autonomous vehicle navigation, the system comprising:
at least one processor included in a server and programmed to:
receive at least one location identifier associated with a directional arrow detected by a vehicle located remotely from the server, the direction arrow being on a surface of a road, and the at least one location identifier being transmitted to the server from the vehicle located remotely from the server;
receive at least one directional indicator associated with the detected directional arrow, the at least one directional indicator being transmitted to the server from the vehicle located remotely from the server;
associate the detected directional arrow with a corresponding road segment;
update an autonomous vehicle road navigation model relative to the corresponding road segment based on the at least one location identifier and based on the at least one directional indicator associated with the detected directional arrow, wherein the autonomous vehicle road navigation model includes at least one target trajectory for a vehicle to follow along the corresponding road segment, the at least one target trajectory being related to the detected directional arrow; and
automatically distribute the updated autonomous vehicle road navigation model to a plurality of autonomous vehicles via one or more networks through one or more wireless communication paths, wherein at least one of the plurality of autonomous vehicles is configured to implement at least one navigational action based on the updated autonomous vehicle road navigation model.

### Claim 2 (depends on 1)

The system of claim 1, wherein the at least one location identifier is determined based on:
acquisition, from a camera associated with a host vehicle, of at least one image representative of an environment of the host vehicle;
analysis of the at least one image to detect the directional arrow on the surface of the road in the environment of the host vehicle; and
analysis of the at least one image to determine a position of the detected directional arrow relative to at least one location associated with the host vehicle.

### Claim 3 (depends on 1)

The system of claim 1, wherein the at least one location identifier includes a location in real world coordinates of a point associated with the detected directional arrow.

### Claim 4 (depends on 3)

The system of claim 3, wherein the point coincides with a location associated with a tip of the directional arrow.

### Claim 5 (depends on 3)

The system of claim 3, wherein the point coincides with a location associated with an interior of the directional arrow.

### Claim 6 (depends on 1)

The system of claim 1, wherein the at least one directional indicator includes locations in real world coordinates of two or more points associated with a head of the detected directional arrow.

### Claim 7 (depends on 1)

The system of claim 1, wherein the at least one directional indicator includes a designation of a point associated with the detected directional arrow as a tip of the directional arrow.

### Claim 8 (depends on 1)

The system of claim 1, wherein the at least one directional indicator includes an ordered pair of points associated with the detected directional arrow.

### Claim 9 (depends on 1)

The system of claim 1, wherein the autonomous vehicle road navigation model also includes at least one target trajectory for a vehicle to follow along the corresponding road segment.

### Claim 10 (depends on 1)

The system of claim 1, wherein the at least one processor is further programmed to receive a first communication from a first host vehicle, wherein the first communication includes the at least one location identifier associated with the detected directional arrow, and receive a second communication from a second host vehicle, wherein the second communication includes at least one additional location identifier associated with the detected directional arrow, and wherein the at least one processor is programmed to refine a determination of at least one position associated with the detected directional arrow based on the at least one location identifier received in the first communication from the first host vehicle and based on the at least one location identifier received in the second communication from the second host vehicle.

### Claim 11 (depends on 1)

The system of claim 1, wherein updating of the autonomous vehicle road navigation model relative to the corresponding road segment based on the at least one location identifier and based on the at least one directional indicator associated with the detected direction includes storing at least one indicator of position in real world coordinates and at least one indicator of orientation of the detected directional arrow.

### Claim 12 (depends on 1)

The system of claim 1, wherein the at least one processor is further programmed to forego the update to the autonomous vehicle road navigation model relative to the corresponding road segment based on the at least one location identifier and based on the at least one directional indicator associated with the detected directional arrow if a direction of travel of a host vehicle, from which the at least one location identifier and the at least one directional indicator were received, is not consistent with an orientation of the detected directional arrow determined based, at least in part, on the at least one directional indicator associated with the detected directional arrow.

### Claim 13

A method for mapping a directional arrow for use in autonomous vehicle navigation, the method comprising:
receiving, by a server, at least one location identifier associated with a directional arrow detected by a vehicle located remotely from the server, the direction arrow being on a surface of a road, and the at least one location identifier being transmitted to the server from the vehicle located remotely from the server;
receiving, by the server, at least one directional indicator associated with the detected directional arrow, the at least one directional indicator being transmitted to the server from the vehicle located remotely from the server;
associating, by the server, the detected directional arrow with a corresponding road segment;
updating, by the server, an autonomous vehicle road navigation model relative to the corresponding road segment based on the at least one location identifier and based on the at least one directional indicator associated with the detected directional arrow, wherein the autonomous vehicle road navigation model includes at least one target trajectory for a vehicle to follow along the corresponding road segment, the at least one target trajectory being related to the detected directional arrow; and
automatically distributing, by the server, the updated autonomous vehicle road navigation model to a plurality of autonomous vehicles via one or more networks through one or more wireless communication paths, wherein at least one of the plurality of autonomous vehicles is configured to implement at least one navigational action based on the updated autonomous vehicle road navigation model.

### Claim 14 (depends on 13)

The method of claim 13, wherein the at least one location identifier is determined based on:
acquisition, from a camera associated with a host vehicle, of at least one image representative of an environment of the host vehicle;
analysis of the at least one image to detect the directional arrow on the surface of the road in the environment of the host vehicle; and
analysis of the at least one image to determine a position of the detected directional arrow relative to at least one location associated with the host vehicle.

### Claim 15 (depends on 13)

The method of claim 13, wherein the at least one location identifier includes a location in real world coordinates of a point associated with the detected directional arrow.

### Claim 16 (depends on 13)

The method of claim 13, wherein the at least one directional indicator includes locations in real world coordinates of two or more points associated with a head of the detected directional arrow.

### Claim 17 (depends on 13)

The method of claim 13, wherein the at least one directional indicator includes a designation of a point associated with the detected directional arrow as a tip of the directional arrow.

### Claim 18 (depends on 13)

The method of claim 13, wherein the at least one directional indicator includes an ordered pair of points associated with the detected directional arrow.

### Claim 19

A system for autonomously navigating a host vehicle along a road segment, the system comprising:
at least one processor included in a host vehicle and programmed to:
receive from an image capture device at least one image representative of an environment of the host vehicle;
detect in the at least one image a representation of an arrow present on a road surface;
identify two or more points of the arrow in the at least one image, the two or more points including at least one of a tip point, an exterior vertex point, or an interior vertex point;
determine a pointing direction of the detected arrow based on analysis of the two or more points of the arrow in the at least one image;
determine a location of the detected arrow by analyzing the at least one image;
transmit, to a server located remotely from the host vehicle, the pointing direction and the location of the detected arrow;
determine an autonomous navigational action for the host vehicle based on the determined pointing direction of the detected arrow; and
cause at least one actuator system of the host vehicle to implement the determined autonomous navigational action.

### Claim 20 (depends on 19)

The system of claim 19, wherein the determined autonomous navigational action includes braking of the host vehicle, and wherein the braking is implemented after a determination that the pointing direction of the detected arrow is opposite to a direction of travel of the host vehicle.

### Claim 21 (depends on 19)

The system of claim 19, wherein the determined autonomous navigational action includes steering of the host vehicle, and wherein the steering is implemented after a determination that the pointing direction of the detected arrow is inconsistent with a planned direction of travel for the host vehicle.

### Claim 22 (depends on 19)

The system of claim 19, wherein the determined autonomous navigational action includes steering of the host vehicle in a direction that reduces a difference between a heading direction of the host vehicle and the pointing direction of the detected arrow.

### Claim 23 (depends on 19)

The system of claim 19, wherein the determined autonomous navigational action includes steering of the host vehicle in a direction that increases a difference between a heading direction of the host vehicle and the pointing direction of the detected arrow.

### Claim 24 (depends on 19)

The system of claim 19, wherein the determined autonomous navigational action includes maintaining a current heading direction of the host vehicle.

### Claim 25

A method for autonomously navigating a host vehicle along a road segment, the method comprising:
receiving, by the host vehicle, from an image capture device at least one image representative of an environment of the host vehicle;
detecting, by the host vehicle, in the at least one image a representation of an arrow present on a road surface;
determining, by the host vehicle, a pointing direction of the detected arrow based on analysis of the at least one image;
identifying, by the host vehicle, two or more points of the arrow in the at least one image, the two or more points including at least one of a tip point, an exterior vertex point, or an interior vertex point;
determining, by the host vehicle, a location of the detected arrow by analyzing the two or more points of the arrow in the at least one image;
transmitting, by the host vehicle to a server located remotely from the host vehicle, the pointing direction and the location of the detected arrow;
determining, by the host vehicle, an autonomous navigational action for the host vehicle based on the determined pointing direction of the detected arrow; and
causing, by the host vehicle, at least one actuator system of the host vehicle to implement the determined autonomous navigational action.

### Claim 26 (depends on 25)

The method of claim 25, wherein the determined autonomous navigational action includes braking of the host vehicle, and wherein the braking is implemented after a determination that the pointing direction of the detected arrow is opposite to a direction of travel of the host vehicle.

### Claim 27 (depends on 25)

The method of claim 25, wherein the determined autonomous navigational action includes steering of the host vehicle, and wherein the steering is implemented after a determination that the pointing direction of the detected arrow is inconsistent with a planned direction of travel for the host vehicle.

### Claim 28 (depends on 25)

The method of claim 25, wherein the determined autonomous navigational action includes steering of the host vehicle in a direction that reduces a difference between a heading direction of the host vehicle and the pointing direction of the detected arrow.

### Claim 29 (depends on 25)

The method of claim 25, wherein the determined autonomous navigational action includes steering of the host vehicle in a direction that increases a difference between a heading direction of the host vehicle and the pointing direction of the detected arrow.

### Claim 30 (depends on 25)

The method of claim 25, wherein the determined autonomous navigational action includes maintaining a current heading direction of the host vehicle.

### Claim 31

A system for autonomously navigating a host vehicle along a road segment, the system comprising:
at least one processor included in the host vehicle and programmed to:
receive from a server-based system an autonomous vehicle road navigation model, wherein the autonomous vehicle road navigation model includes a mapped directional arrow associated with the road segment, and wherein the mapped directional arrow includes at least one of an indication of an orientation of the directional arrow or a location of the directional arrow;
determine an autonomous navigational action for the host vehicle based on at least one of the orientation of the directional arrow or the location of the directional arrow; and
cause at least one actuator system of the host vehicle to implement the determined autonomous navigational action,
wherein the at least one processor is further programmed to receive from an image capture device at least one image representative of an environment of the host vehicle, and
wherein the at least one processor is further programmed to determine the autonomous navigational action for the host vehicle based on at least one of the orientation of the directional arrow or the location of the directional arrow included in the autonomous vehicle road navigation model received from the server-based system, even where a representation of the directional arrow does not appear in the at least one image representative of the environment of the host vehicle.

### Claim 32 (depends on 31)

The system of claim 31, wherein the determined autonomous navigational action includes steering of the host vehicle in a direction that reduces a difference between a heading direction of the host vehicle and a pointing direction of the directional arrow.

### Claim 33 (depends on 31)

The system of claim 31, wherein the determined autonomous navigational action includes steering of the host vehicle in a direction that increases a difference between a heading direction of the host vehicle and a pointing direction of the directional arrow.

### Claim 34 (depends on 31)

The system of claim 31, wherein the determined autonomous navigational action includes maintaining a current heading direction of the host vehicle.

### Claim 35 (depends on 31)

The system of claim 31, wherein the autonomous vehicle road navigation model includes a target trajectory for the host vehicle and wherein the at least one processor is further programmed to receive from an image capture device at least one image representative of an environment of the host vehicle and determine a longitudinal position of the host vehicle along the target trajectory based on a location of the mapped directional arrow and based on detection of the directional arrow in the at least one image.

### Claim 36 (depends on 35)

The system of claim 35, wherein the target trajectory is represented as a three-dimensional spline.

### Claim 37 (depends on 31)

The system of claim 31, wherein the mapped directional arrow comprises data indicating the location and direction of an arrow present on a roadway associated with the road segment.

### Claim 38

A method for autonomously navigating a host vehicle along a road segment, the method comprising:
receiving from a server-based system an autonomous vehicle road navigation model, wherein the autonomous vehicle road navigation model includes a mapped directional arrow associated with the road segment, and wherein the mapped directional arrow includes at least one of an indication of an orientation of the directional arrow or a location of the directional arrow;
determining an autonomous navigational action for the host vehicle based on at least one of the orientation of the directional arrow or the location of the directional arrow; and
causing at least one actuator system of the host vehicle to implement the determined autonomous navigational action,
wherein the method further comprises:
receiving from an image capture device at least one image representative of an environment of the host vehicle; and
determining the autonomous navigational action for the host vehicle based on at least one of the orientation of the directional arrow or the location of the directional arrow included in the autonomous vehicle road navigation model received from the server-based system, even where a representation of the directional arrow does not appear in the at least one image representative of the environment of the host vehicle.

### Claim 39 (depends on 38)

The method of claim 38, wherein the determined autonomous navigational action includes maintaining a current heading direction of the host vehicle.

### Claim 40 (depends on 38)

The method of claim 38, wherein the autonomous vehicle road navigation model includes a target trajectory for the host vehicle, and the method further comprises receiving from an image capture device at least one image representative of an environment of the host vehicle and determining a longitudinal position of the host vehicle along the target trajectory based on a location of the mapped directional arrow and based on detection of the directional arrow in the at least one image.