# US 11,079,242 B2 · System and method for determining autonomous vehicle location using incremental image analysis > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,079,242 B2 - Document ID: US11079242B2 - Status: Granted - Kind: B2 - Application number: 16/226,353 - Filed: 2018-12-19 - Published: 2019-07-04 - Granted: 2021-08-03 - Earliest claimed: 2017-12-29 - Assignees: Walmart Apollo, LLC - Applicants: Walmart Apollo, LLC - Inventors: Robert L. Cantrell; David C. Winkle; John J. O'Brien - Web page: https://patentgrep.com/patent/US11079242B2 ## CPC - G06T 7/74: {involving reference images or patches} - B64U 10/14: with four distinct rotor axes, e.g. quadcopters - B64U 20/87: Mounting of imaging devices, e.g. mounting of gimbals - G01C 21/005: {with correlation of navigation data from several sources, e.g. map or contour matching (G01C21/30 takes precedence)} - G01C 21/28: with correlation of data from several navigational instruments - G05D 1/00: Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots (drive control systems specially adapted for autonomous road vehicles B60W60/00) - G06F 16/29: Geographical information databases - G06F 16/583: using metadata automatically derived from the content - G06F 16/9537: Spatial or temporal dependent retrieval, e.g. spatiotemporal queries - B64U 2101/30: for imaging, photography or videography - B64U 2201/00: UAVs characterised by their flight controls - B64U 2201/10: autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS] - G06T 2207/10032: Satellite or aerial image; Remote sensing - G06T 2207/30244: Camera pose - G06T 2207/30252: Vehicle exterior; Vicinity of vehicle - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - B64U: UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR - G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S) - G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Systems, methods, and computer-readable storage media for determining autonomous vehicle location using incremental image analysis. An exemplary method can include identifying an expected position of an autonomous vehicle which is moving, and identifying, an actual position of the autonomous vehicle. The identifying of the actual position occurs by obtaining images of the autonomous vehicle's surroundings, initiating an iterative image comparison of those images to previously stored images within a given geographic radius of the autonomous vehicle, and iteratively extending the radius (and the pictures being compared) until a match is found or until the maximum radius is reached. ## Claims ### Claim 1 A method comprising: identifying, at a central location and via a processor accessing a memory device, an expected position of an autonomous vehicle which is moving; and identifying, via the processor, an actual position of the autonomous vehicle, by: transmitting, from the processor at the central location to the autonomous vehicle, a request for on-board location information, the on-board location information comprising: surrounding images comprising surroundings of the autonomous vehicle; and peer GPS (Global Positioning System) data for proximate peers; receiving, at the central location from the autonomous vehicle and in response to the request, the surrounding images; receiving, at the central location from the autonomous vehicle and in response to the request, the peer GPS data; determining, based on the peer GPS data in the on-board location information, that the actual position of the autonomous vehicle is outside a threshold range of the expected position, to yield a determination; initiating, based on the determination, an iterative image comparison, wherein each iteration of the iterative image comparison: extends a geographic radius extending from the expected position; and compares stored images within the geographic radius to the surrounding images, until the comparison of the stored images to the surrounding images within a given geographic radius yields the actual position with a predetermined certainty. ### Claim 2 (depends on 1) The method of claim 1, wherein the surrounding images comprise a panoramic image with a 360° view around the autonomous vehicle. ### Claim 3 (depends on 1) The method of claim 1, wherein the autonomous vehicle is an aerial drone, and the surrounding images comprise an image of ground beneath the aerial drone. ### Claim 4 (depends on 1) The method of claim 1, wherein the proximate peers comprise a plurality of other autonomous vehicles, and the peer GPS data comprise GPS coordinates of the other autonomous vehicles. ### Claim 5 (depends on 1) The method of claim 1, wherein the identifying of the actual position of the autonomous vehicle occurs in real-time, with the surrounding images are taken by the autonomous vehicle immediately upon receiving the request, and the initiation of the iterative image comparison begins immediately upon receiving the on-board location information. ### Claim 6 (depends on 1) The method of claim 1, wherein each iteration of the iterative image comparison further comprises: filtering landmarks from the stored images which are located outside the geographic radius for a given iteration, to yield filtered images; and wherein the comparing of the stored images to the surrounding images uses the filtered images. ### Claim 7 (depends on 6) The method of claim 6, wherein comparing the stored images to the surrounding images using the filtered images is more computationally efficient than comparing the stored images to an entirety of the surrounding images. ### Claim 8 (depends on 1) The method of claim 1, wherein the on-board location information further comprises distance data corresponding to the surrounding images, the distance data identifying a distance between the autonomous vehicle and an object pictured within the surrounding images. ### Claim 9 A system comprising: a processor; and a computer-readable storage medium having instructions stored which, when executed by the processor, cause the processor to perform operations comprising: identifying, at a central location, an expected position of an autonomous vehicle which is moving; and identifying an actual position of the autonomous vehicle, by: transmitting, from the central location to the autonomous vehicle, a request for on-board location information, the on-board location information comprising: surrounding images comprising surroundings of the autonomous vehicle; and peer GPS (Global Positioning System) data for proximate peers; receiving, at the central location from the autonomous vehicle and in response to the request, the surrounding images; receiving, at the central location from the autonomous vehicle and in response to the request, the peer GPS data; determining, based on the peer GPS data in the on-board location information, that the actual position of the autonomous vehicle is outside a threshold range of the expected position, to yield a determination; initiating, based on the determination, an iterative image comparison, wherein each iteration of the iterative image comparison: extends a geographic radius extending from the expected position; and compares stored images within the geographic radius to the surrounding images, until the comparison of the stored images to the surrounding images within a given geographic radius yields the actual position with a predetermined certainty. ### Claim 10 (depends on 9) The system of claim 9, wherein the surrounding images comprise a panoramic image with a 360° view around the autonomous vehicle. ### Claim 11 (depends on 9) The system of claim 9, wherein the autonomous vehicle is an aerial drone, and the surrounding images comprise an image of ground beneath the aerial drone. ### Claim 12 (depends on 9) The system of claim 9, wherein the proximate peers comprise a plurality of other autonomous vehicles, and the peer GPS data comprise GPS coordinates of the other autonomous vehicles. ### Claim 13 (depends on 9) The system of claim 9, wherein the identifying of the actual position of the autonomous vehicle occurs in real-time, with the surrounding images are taken by the autonomous vehicle immediately upon receiving the request, and the initiation of the iterative image comparison begins immediately upon receiving the on-board location information. ### Claim 14 (depends on 9) The system of claim 9, wherein each iteration of the iterative image comparison further comprises: filtering landmarks from the stored images which are located outside the geographic radius for a given iteration, to yield filtered images; and wherein the comparing of the stored images to the surrounding images uses the filtered images. ### Claim 15 (depends on 14) The system of claim 14, wherein comparing the stored images to the surrounding images using the filtered images is more computationally efficient than comparing the stored images to an entirety of the surrounding images. ### Claim 16 (depends on 9) The system of claim 9, wherein the on-board location information further comprises distance data corresponding to the surrounding images, the distance data identifying a distance between the autonomous vehicle and an object pictured within the surrounding images. ### Claim 17 A non-transitory computer-readable storage medium having instructions stored which, when executed by a computing device, cause the computing device to perform operations comprising: identifying, at a central location, an expected position of an autonomous vehicle which is moving; and identifying an actual position of the autonomous vehicle, by: transmitting, at the central location to the autonomous vehicle, a request for on-board location information, the on-board location information comprising: surrounding images comprising surroundings of the autonomous vehicle; and peer GPS (Global Positioning System) data for proximate peers; receiving, at the central location from the autonomous vehicle and in response to the request, the surrounding images; receiving, at the central location from the autonomous vehicle and in response to the request, the peer GPS data; determining, based on the peer GPS data in the on-board location information, that the actual position of the autonomous vehicle is outside a threshold range of the expected position, to yield a determination; initiating, based on the determination, an iterative image comparison, wherein each iteration of the iterative image comparison: extends a geographic radius extending from the expected position; and compares stored images within the geographic radius to the surrounding images, until the comparison of the stored images to the surrounding images within a given geographic radius yields the actual position with a predetermined certainty. ### Claim 18 (depends on 17) The non-transitory computer-readable storage medium of claim 17, wherein the surrounding images comprise a panoramic image with a 360° view around the autonomous vehicle. ### Claim 19 (depends on 17) The non-transitory computer-readable storage medium of claim 17, wherein the autonomous vehicle is an aerial drone, and the surrounding images comprise an image of ground beneath the aerial drone. ### Claim 20 (depends on 17) The non-transitory computer-readable storage medium of claim 17, wherein the proximate peers comprise a plurality of other autonomous vehicles, and the peer GPS data comprise GPS coordinates of the other autonomous vehicles.