# US 11,308,322 B2 · Locating a client device using ground truth image rendering

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,308,322 B2
- Document ID: US11308322B2
- Status: Granted
- Kind: B2
- Application number: 16/712,829
- Filed: 2019-12-12
- Published: 2020-07-23
- Granted: 2022-04-19
- Earliest claimed: 2019-01-23
- Assignees: Uber Technologies, Inc.
- Applicants: Uber Technologies, Inc.
- Inventors: Aaron Matthew Rogan
- Web page: https://patentgrep.com/patent/US11308322B2

## CPC

- G06Q 10/02: Reservations, e.g. for tickets, services or events
- G01C 21/343: {Calculating itineraries (travelling salesman problem G06Q10/04; optimisation of routes G06Q10/047)}
- G01C 21/3438: {Rendezvous; Ride sharing}
- G01C 21/3602: {Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera}
- G01C 21/3644: {Landmark guidance, e.g. using POIs or conspicuous other objects}
- G01C 21/3647: {Guidance involving output of stored or live camera images or video streams}
- G01C 21/365: {Guidance using head up displays or projectors, e.g. virtual vehicles or arrows projected on the windscreen or on the road itself}
- G01S 5/16: using electromagnetic waves other than radio waves
- G01S 19/48: by combining or switching between position solutions derived from the satellite radio beacon positioning system and position solutions derived from a further system
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/016: {Input arrangements with force or tactile feedback as computer generated output to the user}
- G06F 16/5866: {using information manually generated, e.g. tags, keywords, comments, manually generated location and time information}
- G06F 16/587: using geographical or spatial information, e.g. location
- G06F 16/7837: {using objects detected or recognised in the video content}
- G06F 18/22: Matching criteria, e.g. proximity measures
- G06Q 10/047: Optimisation of routes or paths, e.g. travelling salesman problem
- G06Q 30/0209: {Incentive being awarded or redeemed in connection with the playing of a video game}
- G06Q 30/0235: constrained by time limit or expiration date
- G06Q 50/40: Business processes related to the transportation industry (shipping G06Q10/083)
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
- G06V 20/00: Scenes; Scene-specific elements (control of digital cameras H04N23/60)
- G06V 20/20: in augmented reality scenes
- G06V 20/30: in albums, collections or shared content, e.g. social network photos or video
- G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle
- H04B 1/3827: Portable transceivers
- H04L 67/52: specially adapted for the location of the user terminal
- H04N 23/63: by using electronic viewfinders
- H04N 23/64: {Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image}
- H04W 4/021: Services related to particular areas, e.g. point of interest [POI] services, venue services or geofences
- H04W 4/029: Location-based management or tracking services
- H04W 4/185: {by embedding added-value information into content, e.g. geo-tagging}
- H04W 4/42: for mass transport vehicles, e.g. buses, trains or aircraft
- H04W 4/44: for communication between vehicles and infrastructures, e.g. vehicle-to-cloud [V2C] or vehicle-to-home [V2H]
- G06T 2200/24: involving graphical user interfaces [GUIs]
- G06T 2219/024: Multi-user, collaborative environment
- H04L 67/131: Protocols for games, networked simulations or virtual reality
- H04N 5/272: Means for inserting a foreground image in a background image, i.e. inlay, outlay
- H04W 4/024: Guidance services
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S)
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04B: TRANSMISSION
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72)

## Abstract

Systems and methods are disclosed herein for determining a location of a device, for example, by receiving, from a client device, a rendering of an image captured by the client device. The systems and methods compare the received rendering to entries in a database of renderings, each respective entry including a respective rendering and a respective associated location, and determine, from the comparing whether the received rendering matches a respective rendering included in a respective entry in the database of renderings. The systems and methods, in response to determining that the received rendering matches the respective rendering included in the respective entry, determine the location associated with the matching rendering, and transmit the location to the client device.

## Claims

### Claim 1

A computer-implemented method for determining a location of a device, the method comprising:
receiving, from a client device, a global positioning system (GPS) trace used to identify a pickup location estimate in a geographic region where the client device is located;
determining that the geographic region matches a region determined to be associated with erroneous GPS traces, the region determined to be associated with the erroneous GPS traces based on at least a number of GPS traces that are received from client devices located within vehicles and indicate that the client devices are at locations without road access;
causing display at the client device of a user interface prompt that, when selected, initiates a process for improving the pickup location estimate;
in response to detecting a selection of the user interface prompt, causing display of a prompt to capture, by the client device, an image of a vicinity of the client device;
receiving, from the client device, a rendering of the image captured by the client device;
determining, by comparing the received rendering to entries in a database of renderings, each entry including a respective rendering and a respective associated location, that
the received rendering matches a respective rendering included in a respective entry in the database of renderings; and
transmitting a location associated with the matching rendering to the client device, wherein a visual indication of a location of the client device based on the transmitted location is displayed on a user interface of the client device.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, further comprising causing the user interface of the client device to visually indicate the location of the client device based on the transmitted location by updating the user interface to replace a visual indication of the location estimate with a visual indication of the transmitted location.

### Claim 3 (depends on 1)

The computer-implemented method of claim 1, wherein comparing the received rendering to the entries comprises:
extracting geolocation data from the rendering;
determining a subset of the entries corresponding to the geolocation data; and
limiting the comparing of the received rendering to the subset of the entries.

### Claim 4 (depends on 1)

The computer-implemented method of claim 1, wherein determining whether the received rendering matches the respective entry of the entries comprises:
determining that the received rendering does not completely match any entry of the entries;
in response to determining that the received rendering does not completely match any entry of the entries, determining that a percentage of characteristics of the received rendering match characteristics of the given entry;
determining whether the percentage exceeds a threshold;
in response to determining that the percentage exceeds the threshold, determining that the received rendering matches the given entry; and
in response to determining that the threshold does not exceed the threshold, determining that the received rendering does not match the given entry.

### Claim 5 (depends on 4)

The computer-implemented method of claim 4, further comprising, further in response to determining that the received rendering does not match the given entry, transmitting a prompt to an administrator to add an entry corresponding to the location of the client device.

### Claim 6 (depends on 1)

The computer-implemented method of claim 1, further comprising:
causing the user interface to designate, in a context of the image, a destination to which the user is to travel.

### Claim 7 (depends on 6)

The computer-implemented method of claim 6, wherein the user interface designates the destination by emitting haptic feedback on the client device corresponding to a location to which the user is to travel.

### Claim 8

A non-transitory computer-readable storage medium storing computer program instructions executable by a processor to perform operations for estimating a location of a client device, the operations comprising:
receiving, from a client device, a global positioning system (GPS) trace used to identify a pickup location estimate in a geographic region where the client device is located;
determining that the geographic region matches a region determined to be associated with erroneous GPS traces, the region determined to be associated with the erroneous GPS traces based on at least a number of GPS traces that are received from client devices located within vehicles and indicate that the client devices are at locations without road access;
causing display at the client device of a user interface prompt that, when selected, initiates a process for improving the pickup location estimate;
in response to detecting a selection of the user interface prompt, causing display of a prompt to capture, by the client device, an image of a vicinity of the client device;
receiving, from the client device, a rendering of the image captured by the client device;
determining, by comparing the received rendering to entries in a database of renderings, each entry including a respective rendering and a respective associated location, that
the received rendering matches a respective rendering included in a respective entry in the database of renderings; and
transmitting a location associated with the matching rendering to the client device, wherein a visual indication of a location of the client device based on the transmitted location is displayed on a user interface of the client device.

### Claim 9 (depends on 8)

The non-transitory computer-readable storage medium of claim 8, wherein the operations further comprise causing the user interface of the client device to visually indicate the location of the client device based on the transmitted location by updating the user interface to replace a visual indication of the location estimate with a visual indication of the transmitted location.

### Claim 10 (depends on 8)

The non-transitory computer-readable storage medium of claim 8, wherein comparing the received rendering to the entries comprises:
extracting geolocation data from the rendering;
determining a subset of the entries corresponding to the geolocation data; and
limiting the comparing of the received rendering to the subset of the entries.

### Claim 11 (depends on 8)

The non-transitory computer-readable storage medium of claim 8, wherein determining whether the received rendering matches the respective entry of the entries comprises:
determining that the received rendering does not completely match any entry of the entries;
in response to determining that the received rendering does not completely match any entry of the entries, determining that a percentage of characteristics of the received rendering match characteristics of the given entry;
determining whether the percentage exceeds a threshold;
in response to determining that the percentage exceeds the threshold, determining that the received rendering matches the given entry; and
in response to determining that the threshold does not exceed the threshold, determining that the received rendering does not match the given entry.

### Claim 12 (depends on 11)

The non-transitory computer-readable storage medium of claim 11, the operations further comprising, further in response to determining that the received rendering does not match the given entry, transmitting a prompt to an administrator to add an entry corresponding to the location of the client device.

### Claim 13 (depends on 8)

The non-transitory computer-readable storage medium of claim 8, the operations further comprising causing the user interface to designate, in a context of the image, a destination to which the user is to travel.

### Claim 14 (depends on 13)

The non-transitory computer-readable storage medium of claim 13, wherein the user interface designates the destination by emitting haptic feedback on the client device corresponding to a location to which the user is to travel.

### Claim 15

A system, comprising:
a processor for executing computer program instructions; and
a non-transitory computer-readable storage medium storing computer program instructions executable by the processor to perform operations for estimating a location of a client device, the operations comprising:
receiving, from a client device, a global positioning system (GPS) trace used to identify a pickup location estimate in a geographic region where the client device is located;
determining that the geographic region matches a region determined to be associated with erroneous GPS traces, the region determined to be associated with the erroneous GPS traces based on at least a number of GPS traces that are received from client devices located within vehicles and indicate that the client devices are at locations without road access;
causing display at the client device of a user interface prompt that, when selected, initiates a process for improving the pickup location estimate;
in response to detecting a selection of the user interface prompt, causing display of a prompt to capture, by the client device, an image of a vicinity of the client device;
receiving, from the client device, a rendering of the image captured by the client device;
determining, by comparing the received rendering to entries in a database of renderings, each entry including a respective rendering and a respective associated location, that
the received rendering matches a respective rendering included in a respective entry in the database of renderings; and
transmitting a location associated with the matching rendering to the client device, wherein a visual indication of a location of the client device based on the transmitted location is displayed on a user interface of the client device.

### Claim 16 (depends on 15)

The system of claim 15, wherein the operations further comprise causing the user interface of the client device to visually indicate the location of the client device based on the transmitted location by updating the user interface to replace a visual indication of the location estimate with a visual indication of the transmitted location.

### Claim 17 (depends on 8)

The non-transitory computer-readable storage medium of claim 8, wherein the operations further comprise:
causing display at the user interface of the client device a visual indication of the pickup location estimate; and
receiving, from the client device, user feedback that the pickup location estimate is inaccurate, wherein the region is determined to be associated with the erroneous GPS traces further based on the user feedback.

### Claim 18 (depends on 8)

The non-transitory computer-readable storage medium of claim 8, wherein the user interface prompt is a first user interface prompt, and wherein the operations further comprise causing display at the user interface of the client device a second user interface prompt that, when selected, causes the client device to transmit a confirmation that the transmitted location is accurate.

### Claim 19 (depends on 1)

The computer-implemented method of claim 1, further comprising:
causing display at the user interface of the client device a visual indication of the pickup location estimate; and
receiving, from the client device, user feedback that the pickup location estimate is inaccurate, wherein the region is determined to be associated with the erroneous GPS traces further based on the user feedback.

### Claim 20 (depends on 1)

The computer-implemented method of claim 1, wherein the user interface prompt is a first user interface prompt, and further comprising causing display at the user interface of the client device a second user interface prompt that, when selected, causes the client device to transmit a confirmation that the transmitted location is accurate.