# US 2019/0168748 A1 · Teleassistance data prioritization for self-driving vehicles

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0168748 A1
- Document ID: US20190168748A1
- Status: Published application
- Kind: A1
- Application number: 16/271,558
- Filed: 2019-02-08
- Published: 2019-06-06
- Earliest claimed: 2017-03-07
- Assignees: Uber Technologies, Inc.
- Applicants: Uber Technologies, Inc.
- Inventors: Benjamin Kroop; William Ross; Andrew Heine
- Web page: https://patentgrep.com/patent/US20190168748A1

## CPC

- B60W 60/0027: {using trajectory prediction for other traffic participants}
- B60W 10/04: including control of propulsion units
- B60W 10/18: including control of braking systems
- B60W 10/20: including control of steering systems
- B60W 30/09: Taking automatic action to avoid collision, e.g. braking and steering
- B60W 50/029: Adapting to failures or work around with other constraints, e.g. circumvention by avoiding use of failed parts
- B60W 60/0015: {specially adapted for safety}
- B60W 60/00253: {Taxi operations}
- B60W 60/00276: {for two or more other traffic participants}
- G05D 1/0022: {characterised by the communication link (data switching networks in general H04L12/00)}
- G05D 1/0038: {by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation (images analyzed by a computer and used for automatic navigation G05D1/0246)}
- G05D 1/0231: {using optical position detecting means (position-fixing by using electromagnetic waves other than radio waves, e.g. optical position detecting means G01S5/16)}
- G05D 1/0246: {using a video camera in combination with image processing means}
- G07C 5/008: {communicating information to a remotely located station (transmission systems for measured values G08C)}
- B60W 2050/0215: {Sensor drifts or sensor failures}
- B60W 2420/403: Image sensing, e.g. optical camera
- B60W 2420/408: Radar; Laser, e.g. lidar
- B60W 2552/45: Pedestrian sidewalk
- B60W 2554/00: Input parameters relating to objects
- B60W 2554/20: Static objects
- B60W 2554/4026: Cycles
- B60W 2554/4029: Pedestrians
- B60W 2554/802: Longitudinal distance
- B60W 2710/18: Braking system
- B60W 2710/20: Steering systems
- B60W 2720/106: Longitudinal acceleration
- H04W 84/04: Large scale networks; Deep hierarchical networks
- H04W 84/12: WLAN [Wireless Local Area Networks]
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G07C: TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
- H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72)

## Abstract

A system can analyze a live sensor view of a self-driving vehicle (SDV) in accordance with a safety threshold to detect objects of interest along a route and classify each detected object of interest. When the safety threshold is not met, the system can transmit a teleassistance inquiry using LIDAR data to a backend computing system. When a certainty threshold is not met for an object of interest, the system can transmit a different teleassistance inquiry using image data to the backend computing system.

## Claims

### Claim 1

A system comprising:
one or more processors; and
a memory storing instructions that, when executed by the one or more processors, cause the system to:
analyze a live sensor view of a self-driving vehicle (SDV) in accordance with a safety threshold to (i) detect objects of interest along a route, and (ii) classify each respective object of the detected objects of interest;
determine that the safety threshold is not met;
in response to determining that the safety threshold is not met, transmit a first teleassistance inquiry to a backend computing system, the first teleassistance inquiry comprising LIDAR data;
determine that a classification of one or more of the detected objects of interest does not satisfy a certainty threshold;
in response to determining that the classification of the one or more of the detected objects of interest does not satisfy the certainty threshold, transmit a second teleassistance inquiry to the backend computing system, the second teleassistance inquiry comprising image data that includes the one or more detected objects of interest;
receive a resolution response from the backend computing system; and
provide a control system of the SDV with data corresponding to the resolution response to enable the SDV to proceed in accordance with the resolution response.

### Claim 2 (depends on 1)

The system of claim 1, wherein the safety threshold not being met corresponds to an obstruction in the live sensor view.

### Claim 3 (depends on 1)

The system of claim 1, wherein the executed instructions cause the system to transmit LIDAR data and not image data when the safety threshold is not met.

### Claim 4 (depends on 1)

The system of claim 1, wherein the executed instructions cause the system to transmit image data and not LIDAR data when the certainty threshold is not met.

### Claim 5 (depends on 1)

The system of claim 1, wherein the executed instructions further cause the system to”
monitor bandwidth of each of a plurality of wireless communication modules of the SDV to determine a highest bandwidth communication module;
wherein the executed instructions cause the system to transmit each of the first and the second teleassistance inquiries to the backend computing system using the highest bandwidth communication module.

### Claim 6 (depends on 5)

The system of claim 5, wherein the executed instructions further cause the system to:
duplicate transmission of each of the first and the second teleassistance inquiries by transmitting the first and the second teleassistance inquiries over a second communication module from the plurality of wireless communication modules.

### Claim 7 (depends on 6)

The system of claim 6, wherein the highest bandwidth communication module and the second communication module both comprise Long Term Evolution (LTE) communications modules.

### Claim 8 (depends on 6)

The system of claim 6, wherein the highest bandwidth communication module comprises a LTE communication module and the second communication module comprises a Wi-Fi module.

### Claim 9 (depends on 1)

The system of claim 1, wherein the first and the second teleassistance inquiries include telemetry data for the SDV.

### Claim 10 (depends on 1)

The system of claim 1, wherein the first and the second teleassistance inquiries include localization information for the SDV.

### Claim 11 (depends on 1)

The system of claim 1, wherein the first and the second teleassistance inquiries include diagnostics data for the SDV.

### Claim 12 (depends on 1)

The system of claim 1, wherein the first teleassistance inquiry is different from the second teleassistance inquiry.

### Claim 13

A non-transitory computer readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to:
analyze a live sensor view of a self-driving vehicle (SDV) in accordance with a safety threshold to (i) detect objects of interest along a route, and (ii) classify each respective object of the detected objects of interest;
determine that the safety threshold is not met;
in response to determining that the safety threshold is not met, transmit a first teleassistance inquiry to a backend computing system, the first teleassistance inquiry comprising LIDAR data;
determine that a classification of one or more of the detected objects of interest does not satisfy a certainty threshold;
in response to determining that the classification of the one or more of the detected objects of interest does not satisfy the certainty threshold, transmit a second teleassistance inquiry to the backend computing system, the second teleassistance inquiry comprising image data that includes the one or more detected objects of interest;
receive a resolution response from the backend computing system; and
provide a control system of the SDV with data corresponding to the resolution response to enable the SDV to proceed in accordance with the resolution response.

### Claim 14 (depends on 13)

The non-transitory computer readable medium of claim 13, wherein the safety threshold not being met corresponds to an obstruction in the live sensor view.

### Claim 15 (depends on 13)

The non-transitory computer readable medium of claim 13, wherein the executed instructions cause the system to transmit LIDAR data and not image data when the safety threshold is not met.

### Claim 16 (depends on 13)

The non-transitory computer readable medium of claim 13, wherein the executed instructions cause the system to transmit image data and not LIDAR data when the certainty threshold is not met.

### Claim 17 (depends on 13)

The non-transitory computer readable medium of claim 13, wherein the executed instructions further cause the system to”
monitor bandwidth of each of a plurality of wireless communication modules of the SDV to determine a highest bandwidth communication module;
wherein the executed instructions cause the system to transmit each of the first and the second teleassistance inquiries to the backend computing system using the highest bandwidth communication module.

### Claim 18 (depends on 17)

The non-transitory computer readable medium of claim 17, wherein the executed instructions further cause the system to:
duplicate transmission of each of the first and the second teleassistance inquiries by transmitting the first and the second teleassistance inquiries over a second communication module from the plurality of wireless communication modules.

### Claim 19 (depends on 18)

The non-transitory computer readable medium of claim 18, wherein the highest bandwidth communication module and the second communication module both comprise Long Term Evolution (LTE) communications modules.

### Claim 20

A computer-implemented method performed by one or more processors and comprising:
analyzing a live sensor view of a self-driving vehicle (SDV) in accordance with a safety threshold to (i) detect objects of interest along a route, and (ii) classify each respective object of the detected objects of interest;
determining that the safety threshold is not met;
in response to determining that the safety threshold is not met, transmitting a first teleassistance inquiry to a backend computing system, the first teleassistance inquiry comprising LIDAR data;
determining that a classification of one or more of the detected objects of interest does not satisfy a certainty threshold;
in response to determining that the classification of the one or more of the detected objects of interest does not satisfy the certainty threshold, transmitting a second teleassistance inquiry to the backend computing system, the second teleassistance inquiry comprising image data that includes the one or more detected objects of interest;
receiving a resolution response from the backend computing system; and
providing a control system of the SDV with data corresponding to the resolution response to enable the SDV to proceed in accordance with the resolution response.
