# US 10,871,782 B2 · Autonomous vehicle control using submaps > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,871,782 B2 - Document ID: US10871782B2 - Status: Granted - Kind: B2 - Application number: 15/640,289 - Filed: 2017-06-30 - Published: 2018-01-04 - Granted: 2020-12-22 - Earliest claimed: 2016-07-01 - Assignees: Uatc, LLC - Applicants: Uatc, LLC - Inventors: Adam Milstein; Brett Browning - Web page: https://patentgrep.com/patent/US10871782B2 ## CPC - G01C 21/3815: {Road data} - B60W 30/095: Predicting travel path or likelihood of collision - G01C 21/28: with correlation of data from several navigational instruments - G01C 21/3602: {Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera} - G01C 21/3811: {Point data, e.g. Point of Interest [POI]} - G01C 21/3819: {Road shape data, e.g. outline of a route} - G01C 21/3822: {Road feature data, e.g. slope data} - G01C 21/3841: {Data obtained from two or more sources, e.g. probe vehicles} - G01C 21/3848: {Data obtained from both position sensors and additional sensors} - G01C 21/3867: {Geometry of map features, e.g. shape points, polygons or for simplified maps} - G01C 21/3878: {Hierarchical structures, e.g. layering} - G05D 1/00: Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots (drive control systems specially adapted for autonomous road vehicles B60W60/00) - G05D 1/0212: {with means for defining a desired trajectory (involving a plurality of land vehicles G05D1/0287)} - G05D 1/0231: {using optical position detecting means (position-fixing by using electromagnetic waves other than radio waves, e.g. optical position detecting means G01S5/16)} - G05D 1/024: {in combination with a laser (lasers per se H01S)} - G05D 1/0246: {using a video camera in combination with image processing means} - G05D 1/0251: {extracting 3D information from a plurality of images taken from different locations, e.g. stereo vision (stereoscopic image analysis H04N13/00; depth recovery from images G06T7/593)} - G05D 1/0274: {using mapping information stored in a memory device (navigation using map-matching G01C21/30)} - G05D 1/0276: {using signals provided by a source external to the vehicle (involving a plurality of vehicles G05D1/0287; automatically controlling vehicle speed responsive to externally generated signals B60K31/0058)} - G06T 7/33: using feature-based methods - G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80) - G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle - G06V 20/588: {Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road} - H04L 67/52: specially adapted for the location of the user terminal - G06T 2207/10012: Stereo images - G06T 2207/30252: Vehicle exterior; Vicinity of vehicle - H04L 67/12: specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks - G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S) - B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT - G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract A system to use submaps to control operation of a vehicle is disclosed. A storage system may be provided with a vehicle to store a collection of submaps that represent a geographic area where the vehicle may be driven. A programmatic interface may be provided to receive submaps and submap updates independently of other submaps. ## Claims ### Claim 1 A computer-implemented method comprising: retrieving, by one or more computing devices physically located onboard an autonomous vehicle, one or more submaps of a plurality of submaps representing an area of a road network for a geographic region; detecting, by the one or more computing devices, a submap change condition, wherein the submap change condition comprises one or more of an environmental condition, an event occurrence, or a submap update; retrieving, by the one or more computing devices, based at least in part on the submap change condition, one or more alternative submaps of the plurality of submaps, wherein the one or more alternative submaps represent the same area of the road network for the geographic region as the one or more submaps; determining, by the one or more computing devices and based at least in part on a comparison of contemporaneous sensor data of the autonomous vehicle with corresponding sensor data previously stored in association with the one or more alternative submaps, a location and pose of the autonomous vehicle within the area of the road network for the geographic region; and controlling, by the one or more computing devices and based at least in part on the location and pose of the autonomous vehicle, one or more operations of the autonomous vehicle. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, wherein determining the location and pose of the autonomous vehicle comprises comparing one or more objects or features of a scene of the area as represented by the contemporaneous sensor data with one or more objects or features of a previous scene of the area as represented by the corresponding sensor data to determine one or more of a spatial or geometric differential between the scene of the area as represented by the contemporaneous sensor data and the previous scene of the area as represented by the corresponding sensor data. ### Claim 3 (depends on 1) The computer-implemented method of claim 1, wherein: the contemporaneous sensor data comprises data, captured by one or more image sensors of the autonomous vehicle, representing a scene of the area; the corresponding sensor data comprises data, captured by one or more corresponding image sensors, representing a previous scene of the area; and determining the location and pose of the autonomous vehicle comprises comparing the data representing the scene with the data representing the previous scene. ### Claim 4 (depends on 1) The computer-implemented method of claim 1, wherein: the contemporaneous sensor data comprises data, captured by one or more light detection and ranging (LIDAR) sensors of the autonomous vehicle, representing a scene of the area; the corresponding sensor data comprises data, captured by one or more corresponding LIDAR sensors, representing a previous scene of the area; and determining the location and pose of the autonomous vehicle comprises comparing the data representing the scene with the data representing the previous scene. ### Claim 5 (depends on 1) The computer-implemented method of claim 1, wherein: the contemporaneous sensor data comprises data representing a point cloud representing a scene of the area; the corresponding sensor data comprises data representing a point cloud representing a previous scene of the area; and determining the location and pose of the autonomous vehicle comprises comparing the data representing the point cloud representing the scene with the data representing the point cloud representing the previous scene. ### Claim 6 (depends on 5) The computer-implemented method of claim 5, wherein the data representing the point cloud representing the scene represents a point cloud spanning radially from a reference location in front of the autonomous vehicle. ### Claim 7 (depends on 5) The computer-implemented method of claim 5, wherein the data representing the point cloud representing the scene represents a point cloud corresponding to one or more spaces alongside of the autonomous vehicle. ### Claim 8 (depends on 5) The computer-implemented method of claim 5, wherein the data representing the point cloud representing the scene represents a point cloud corresponding to one or more spaces behind the autonomous vehicle. ### Claim 9 (depends on 5) The computer-implemented method of claim 5, wherein the method comprises: identifying, by the one or more computing devices, a plurality of different point clouds representing the previous scene; and selecting, by the one or more computing devices and from amongst the plurality of different point clouds, the point cloud representing the previous scene. ### Claim 10 (depends on 9) The computer-implemented method of claim 9, wherein selecting the point cloud representing the previous scene comprises selecting the point cloud representing the previous scene based at least in part on one or more of: a lighting condition associated with the previous scene; a weather condition associated with the previous scene; a time of day associated with the previous scene; or a season associated with the previous scene. ### Claim 11 (depends on 1) The computer-implemented method of claim 1, wherein determining the location and pose of the autonomous vehicle comprises determining one or more of: a lane the autonomous vehicle is using; a distance of the autonomous vehicle from an edge of a road of the road network; a distance of the autonomous vehicle from an edge of a lane of a road of the road network; or a distance of travel from a point of reference for the one or more alternative submaps. ### Claim 12 (depends on 1) The computer-implemented method of claim 1, wherein determining the location and pose of the autonomous vehicle comprises determining a location coordinate of the autonomous vehicle with respect to a particular submap of the one or more alternative submaps. ### Claim 13 (depends on 1) The computer-implemented method of claim 1, wherein determining the location and pose of the autonomous vehicle comprises determining an orientation of the autonomous vehicle with respect to a particular road segment of the road network. ### Claim 14 (depends on 1) The computer-implemented method of claim 1, wherein determining the location and pose of the autonomous vehicle comprises determining one or more of the location or pose of the autonomous vehicle based at least in part on one or more of a priority or weight indicating one or more of reliability or effectiveness of the corresponding sensor data for purposes of determining the one or more of the location or pose of the autonomous vehicle. ### Claim 15 (depends on 1) The computer-implemented method of claim 1, wherein determining the location and pose of the autonomous vehicle comprises comparing the contemporaneous sensor data with the corresponding sensor data to determine one or more objects or features of the area that can form a basis for one or more of geometric or spatial comparison. ### Claim 16 (depends on 1) The computer-implemented method of claim 1, wherein determining the location and pose of the autonomous vehicle comprises determining one or more of the location or pose of the autonomous vehicle with respect to a location at which the corresponding sensor data was captured. ### Claim 17 An autonomous vehicle comprising: one or more processors; and a memory storing instructions that when executed by the one or more processors cause the autonomous vehicle to perform operations comprising: retrieving one or more submaps of a plurality of submaps representing an area of a road network for a geographic region; detecting a submap change condition, wherein the submap change condition comprises one or more of an environmental condition, and event occurrence, or a submap update; retrieving, based at least in part on the submap change condition, one or more alternative submaps of the plurality of submaps referencing data representing a point cloud representing a previous scene of the same area of the road network for the geographic region represented by the one or more submaps; receiving sensor data representing a point cloud representing a contemporaneous scene of the area of the road network; and determining, based at least in part on a comparison of the data representing the point cloud representing the previous scene with the data representing the point cloud representing the contemporaneous scene, one or more of a location or pose of the autonomous vehicle. ### Claim 18 (depends on 17) The autonomous vehicle of claim 17, wherein determining the one or more of the location or pose of the autonomous vehicle comprises comparing one or more objects or features of the previous scene as represented by the one or more alternative submaps with one or more objects or features of the contemporaneous scene as represented by the sensor data to determine one or more of a spatial or geometric differential between the previous scene as represented by the one or more alternative submaps and the contemporaneous scene as represented by the sensor data. ### Claim 19 (depends on 17) The autonomous vehicle of claim 17, wherein: the sensor data representing the point cloud representing the contemporaneous scene comprises data generated based at least in part on data captured by one or more image sensors of the autonomous vehicle; and the data representing the point cloud representing the previous scene comprises data generated based at least in part on data captured by one or more corresponding image sensors. ### Claim 20 (depends on 17) The autonomous vehicle of claim 17, wherein: the sensor data representing the point cloud representing the contemporaneous scene comprises data generated based at least in part on data captured by one or more light detection and ranging (LIDAR) sensors of the autonomous vehicle; and the data representing the point cloud representing the previous scene comprises data generated based at least in part on data captured by one or more corresponding LIDAR sensors. ### Claim 21 One or more non-transitory computer-readable media comprising instructions that when executed by one or more computing devices physically located onboard an autonomous vehicle cause the one or more computing devices to perform operations comprising: retrieving one or more submaps of a plurality of submaps representing an area of a road network for a geogrpahic region; detecting a submap change condition, wherein the submap change condition comprises one or more of an environmental condition, an event occurrence, or a submap update; retrieving, based at least in part on the submap change condition, one or more alternative submaps of the plurality of submaps referencing data representing a previous scene of the same area of the road network for the geographic region as the one or more submaps; receiving sensor data representing a contemporaneous scene of the area; and determining one or more of a location or pose of the autonomous vehicle by comparing one or more objects or features of the previous scene as represented by the one or more alternative submaps with one or more objects or features of the contemporaneous scene as represented by the sensor data to determine one or more of a spatial or geometric differential between the previous scene as represented by the one or more alternative submaps and the contemporaneous scene as represented by the sensor data. ### Claim 22 (depends on 21) The one or more non-transitory computer-readable media of claim 21, wherein: the data representing the previous scene comprises data representing a point cloud representing the previous scene; the sensor data representing the contemporaneous scene comprises data representing a point cloud representing the contemporaneous scene; and the comparing comprises comparing the data representing the point cloud representing the previous scene with the data representing the point cloud representing the contemporaneous scene. ### Claim 23 (depends on 21) The one or more non-transitory computer-readable media of claim 21, wherein: the sensor data representing the contemporaneous scene comprises data generated based at least in part on data captured by one or more image sensors of the autonomous vehicle; and the data representing the previous scene comprises data generated based at least in part on data captured by one or more corresponding image sensors. ### Claim 24 (depends on 21) The one or more non-transitory computer-readable media of claim 21, wherein: the sensor data representing the contemporaneous scene comprises data generated based at least in part on data captured by one or more light detection and ranging (LIDAR) sensors of the autonomous vehicle; and the data representing the previous scene comprises data generated based at least in part on data captured by one or more corresponding LIDAR sensors.