Back to results
US 12,736,961 B1Granted
Road sign verification in autonomous vehicle remote teleassist system
AssigneeAurora Operations, Inc.
InventorsAndreas Hansen, Niels Joubert, Brian Taylor
Life of this patent
Estimating the term…
Abstract
A teleassist system may be used to assist with verifying road signs encountered by an autonomous vehicle, particularly road signs that may be new and/or temporary in nature, e.g., within construction zones, and thus not maintained in a map database utilized by the autonomous vehicle. Detection of an unmapped road sign by an autonomous vehicle, for example, may be used to initiate a teleassist session with a remote teleassist operator to facilitate autonomous vehicle compliance with the unmapped road sign.
Claims18
An autonomous vehicle control system comprising one or more processors configured to operate an autonomous vehicle in an environment by:
- detecting a road sign in the environment from sensor data captured by a sensor disposed on the autonomous vehicle while the autonomous vehicle is operating on a roadway;
- while the autonomous vehicle is operating on the roadway, accessing a database storing sign data for a plurality of road signs in the environment within which the autonomous vehicle is operated to determine that the detected road sign is not stored in the database;
- and
- while the autonomous vehicle is operating on the roadway, and in response to determining that the detected road sign is not stored in the database, initiating a teleassist session with a remote teleassist system to receive remote teleassist operator input that suggests a vehicle control operation to control operation of the autonomous vehicle to comply with the detected road sign;
- wherein the one or more processors are configured to communicate a notification for the detected road sign to the remote teleassist system during the teleassist session to provide situational awareness about the detected road sign to a teleassist operator of the remote teleassist system;
- wherein the detected road sign is a first detected road sign, the notification is a first notification and the one or more processors are further configured to detect a second road sign in the environment proximate the first detected road sign, and to communicate a sequence of the first notification and a second notification for the second detected road sign to the remote teleassist system during the teleassist session.
3 more elements
A method of operating an autonomous vehicle in an environment, comprising:
- detecting a road sign in the environment from sensor data captured by a sensor disposed on the autonomous vehicle while the autonomous vehicle is operating on a roadway;
- while the autonomous vehicle is operating on the roadway, accessing a database storing sign data for a plurality of road signs in the environment within which the autonomous vehicle is operated to determine that the detected road sign is not stored in the database;
- while the autonomous vehicle is operating on the roadway, and in response to determining that the detected road sign is not stored in the database, initiating a teleassist session with a remote teleassist system to receive remote teleassist operator input that suggests a vehicle control operation to control operation of the autonomous vehicle to comply with the detected road sign;
- and
- communicating a notification for the detected road sign to the remote teleassist system during the teleassist session to provide situational awareness about the detected road sign to a teleassist operator of the remote teleassist system;
- wherein the detected road sign is a first detected road sign, the method further comprises detecting a second road sign in the environment proximate the first detected road sign, and the notification is an aggregated notification for both of the first and second detected road signs.
3 more elements
A teleassist system that is remote to an autonomous vehicle operating in an environment, the teleassist system comprising one or more processors configured to:
- create a teleassist session with the autonomous vehicle in response to receiving a request from the autonomous vehicle triggered in response to the autonomous vehicle detecting a road sign in the environment from sensor data captured by a sensor disposed on the autonomous vehicle while the autonomous vehicle is operating on a roadway and determined, while the autonomous vehicle is operating on the roadway, to be an unmapped road sign that is not stored in a database of the autonomous vehicle;
- provide a teleassist operator with situational awareness information associated with the detected road sign using a user interface;
- and
- while the autonomous vehicle is operating on the roadway, and in response to user input from the teleassist operator through the user interface, communicate to the autonomous vehicle teleassist operator input that suggests a vehicle control operation to control operation of the autonomous vehicle to comply with the detected road sign;
- wherein the one or more processors are configured to receive a notification for the detected road sign from the autonomous vehicle during the teleassist session to provide situational awareness about the detected road sign to the teleassist operator;
- wherein the detected road sign is a first detected road sign, the notification is a first notification and the one or more processors are further configured to receive a sequence of the first notification and a second notification for a second road sign from the autonomous vehicle during the teleassist session, the second road sign detected by the autonomous vehicle in the environment proximate the first detected road sign.
3 more elements
Dependencies are inferred from the claim text.