US 2021/0240195 A1Published application

Systems and methods for utilizing images to determine the position and orientation of a vehicle

AssigneeAmazon Technologies, Inc.

InventorsJohn Alan Atherton, Aziz Umit Batur, Kishor Dilip Bhalerao, Haresh Karnan, Harshavardhan Shirolkar

Life of this patent

Estimating the term…

Loading patent…

Abstract

Described are systems and methods to utilize images to determine the position and/or orientation of a vehicle (e.g., an autonomous ground vehicle) operating in an unstructured environment (e.g., environments such as sidewalks which are typically absent lane markings, road markings, etc.). The described systems and methods can determine the vehicle's position and orientation based on an alignment of annotated images captured during operation of the vehicle with a known annotated reference map. The translation and rotation applied to obtain alignment of the annotated images with the known annotated reference map can provide the position and the orientation of the vehicle.

Claims20

Dependent claims

A system, comprising:

at least one computing system including a first processor and first program instructions that, when executed by the first processor, cause the first processor to at least:
receive map information of an unstructured environment;
construct an overhead two dimensional representation of the unstructured environment based on the map information;
identify a first plurality of environmental features based on the overhead two dimensional representation of the unstructured environment using at least one of a first line segment detector, a first segmented semantic map, or a first trained deep learning system;
and
construct an annotated overhead reference map of the unstructured environment, the annotated overhead reference map including a first plurality of line features corresponding to the first plurality of environmental features;
an autonomous ground vehicle operating in the unstructured environment, the autonomous ground vehicle including:
at least one image sensor;
and
a control system including a second processor and second program instructions that, when executed by the second processor, cause the second processor to at least:
receive at least one image of the unstructured environment from the at least one image sensor;
construct an overhead annotated image of the unstructured environment based on the at least one image of the unstructured environment, the annotated image including a plurality of second line features corresponding to a second plurality of identified environmental features;
determine an alignment of the annotated image to the annotated overhead reference map;
determine a relative translation and a relative rotation utilized in determining the alignment of the annotated image to the annotated overhead reference map;
and
determine at least one of a position and an orientation of the autonomous ground vehicle based on at least one of the relative translation and the relative rotation.

13 more elements

A computer-implemented method, comprising:

constructing an overhead annotated reference map of an unstructured environment, the overhead annotated reference map including a first plurality of line features corresponding to a first plurality of environmental features;
constructing an overhead annotated image of the unstructured environment from a first plurality of perspective view images, the overhead annotated image including a second plurality of line features corresponding to a second plurality of environmental features;
aligning the overhead annotated image of the unstructured environment to the overhead annotated reference map of the unstructured environment;
and
determining at least one of a position and an orientation of an autonomous ground vehicle based at least in part on the aligning of the overhead annotated image of the unstructured environment to the overhead annotated reference map of the unstructured environment.

2 more elements

An autonomous ground vehicle, comprising:

at least one image sensor to capture at least one perspective view image;
at least one positional sensor to provide odometry information;
and
a control system including a processor and program instructions that, when executed by the processor, cause the processor to at least:
receive an overhead annotated reference map having a first plurality of line features of an unstructured environment, the first plurality of line features corresponding to a plurality of environmental features in the unstructured environment,
construct, based at least in part on the at least one perspective view image, an overhead annotated image of the unstructured environment having a second plurality of line features;
determine a translation and a rotation to align the overhead annotated image to the overhead annotated reference map;
and
determine at least one of a position and an orientation of the autonomous ground vehicle based at least in part on the translation and the rotation.

6 more elements

Dependencies are inferred from the claim text.

Facts

Application: 16/779,380Filed 31 Jan 2020
Published: 5 Aug 2021
Term ends: Not estimatedThe source records don’t support an estimate.

Classification17 codes

G01C 21/1656
Combined with non-inertial navigation instruments: with passive imaging devices, e.g. cameras
G01C 21/28
Specially adapted for navigation in a road network: with correlation of data from several navigational instruments
G01C 21/30
With correlation of data from several navigational instruments: map- or contour-matching
G01C 21/3602
Input/output arrangements for on-board computers: input other than that of destination using image analysis, e.g. detection of road signs, lanes,…

Family

US 11,579,622 B2Related patent
US 2021/0240195 A1This patent

From the US filing history. Foreign family members aren’t listed.

Timeline

Citations

Open elsewhere

Building an agent or pulling many patents? Use the API or MCP server instead of scraping pages. Get an API key