# US 2025/0085706 A1 · Assisted Perception for Autonomous Vehicles > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2025/0085706 A1 - Document ID: US20250085706A1 - Status: Published application - Kind: A1 - Application number: 18/953,500 - Filed: 2024-11-20 - Published: 2025-03-13 - Earliest claimed: 2013-11-27 - Assignees: Waymo LLC - Applicants: Waymo LLC - Inventors: David Ian Ferguson; Wan-Yen Lo; Nathaniel Fairfield - Web page: https://patentgrep.com/patent/US20250085706A1 ## CPC - G05D 1/0246: {using a video camera in combination with image processing means} - B60W 40/04: Traffic conditions - B60W 60/0053: {from vehicle to occupant} - G05D 1/00: Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots (drive control systems specially adapted for autonomous road vehicles B60W60/00) - G05D 1/0022: {characterised by the communication link (data switching networks in general H04L12/00)} - G05D 1/226: Communication links with the remote-control arrangements - G05D 1/249: from positioning sensors located off-board the vehicle, e.g. from cameras - G05D 1/81: Handing over between on-board automatic and on-board manual control - G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle - G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads - G07C 5/008: {communicating information to a remotely located station (transmission systems for measured values G08C)} - B60W 2050/0075: {Automatic parameter input, automatic initialising or calibrating means} - B60W 2050/0215: {Sensor drifts or sensor failures} - B60W 2420/403: Image sensing, e.g. optical camera - B60W 2556/45: External transmission of data to or from the vehicle - H04L 67/125: involving control of end-device applications over a network - G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES - B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G07C: TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract Disclosed herein are systems and methods for providing supplemental identification abilities to an autonomous vehicle system. The sensor unit of the vehicle may be configured to receive data indicating an environment of the vehicle, while the control system may be configured to operate the vehicle. The vehicle may also include a processing unit configured to analyze the data indicating the environment to determine at least one object having a detection confidence below a threshold. Based on the at least one object having a detection confidence below a threshold, the processor may communicate at least a subset of the data indicating the environment for further processing. The vehicle is also configured to receive an indication of an object confirmation of the subset of the data. Based on the object confirmation of the subset of the data, the processor may alter the control of the vehicle by the control system. ## Claims ### Claim 1 A method comprising: detecting, by a first computing device and based on sensor data from one or more sensors coupled to a vehicle, an unusual event in an environment of the vehicle; determining an action to perform in response to detecting the unusual event; based on determining a confidence level associated with the vehicle performing the action is below a confidence threshold, transmitting, by the first computing device and to a second computing device, a request for assistance and the sensor data used to detect the unusual event, wherein the second computing device is positioned remote from the vehicle; receiving, at the first computing device, a response from the second computing device; and causing the vehicle to perform a control strategy generated based on the response. ### Claim 2 (depends on 1) The method of claim 1, wherein the response includes a classification for one or more objects corresponding to the unusual event in the environment; and wherein the method further comprises: generating the control strategy based on the classification for the one or more objects corresponding to the unusual event in the environment. ### Claim 3 (depends on 1) The method of claim 1, wherein determining the action to perform comprises: determining a trajectory for the vehicle to use to navigate the unusual event; and wherein transmitting the request for assistance and the sensor data used to detect the unusual event to the second computing device further comprises: transmitting the trajectory determined for the vehicle to the computing device. ### Claim 4 (depends on 3) The method of claim 3, further comprising: determining a given confidence level associated with the trajectory determined for the vehicle; and providing data representing the given confidence level to the second computing device. ### Claim 5 (depends on 3) The method of claim 3, wherein receiving the response from the second computing device comprises: receiving data from the second computing device, the data representing one or more modifications to the trajectory; and wherein the method further comprises: generating the control strategy based on the data. ### Claim 6 (depends on 1) The method of claim 1, wherein detecting the unusual event in the environment comprises: detecting a person potentially directing traffic. ### Claim 7 (depends on 6) The method of claim 6, wherein transmitting the request for assistance and the sensor data used to detect the unusual event comprises: transmitting a video stream showing the detected person to the second computing device. ### Claim 8 (depends on 7) The method of claim 7, wherein receiving the response from the second computing device comprises: receiving an operator input provided based on the video stream. ### Claim 9 (depends on 1) The method of claim 1, wherein detecting the unusual event in the environment comprises: detecting an obstacle blocking a lane of a two-lane road. ### Claim 10 (depends on 1) The method of claim 1, further comprising: responsive to detecting the unusual event in the environment, causing the vehicle to stop and refrain from moving until receiving the response from the second computing device. ### Claim 11 (depends on 1) The method of claim 1, wherein detecting the unusual event in the environment comprises: determining that a first confidence associated with classifying a first object in the environment and a second confidence associated with classifying a second object in the environment are both below the confidence threshold. ### Claim 12 (depends on 1) The method of claim 1, wherein detecting the unusual event in the environment comprises: detecting a difference between map data corresponding to the environment and sensor data representing the environment. ### Claim 13 (depends on 1) The method of claim 1, wherein detecting the unusual event in the environment comprises: obtaining, from one or more microphones coupled to the vehicle, audio data corresponding to the unusual event in the environment; and wherein transmitting the request for assistance and the sensor data used to detect the unusual event comprises: transmitting the audio data to the second computing device. ### Claim 14 (depends on 1) The method of claim 1, wherein the confidence level is related to confirming a presence of a potential object in the environment, and wherein the second computing device is computationally more powerful than the first computing device. ### Claim 15 (depends on 1) The method of claim 1, wherein receiving the response from the second computing device comprises: receiving the response comprising data representing a classification for the unusual event. ### Claim 16 A system comprising: a vehicle comprising a plurality of sensors; and a first computing device coupled to the vehicle, wherein the first computing device is configured to: detect, based on sensor data from one or more sensors of the plurality of sensors, an unusual event in an environment of the vehicle; determine an action to perform in response to detecting the unusual event; based on determining a confidence level associated with the vehicle performing the action is below a confidence threshold, transmit, to a second computing device, a request for assistance and the sensor data used to detect the unusual event, wherein the second computing device is positioned remote from the vehicle; receive a response from the second computing device; and cause the vehicle to perform a control strategy generated based on the response. ### Claim 17 (depends on 16) The system of claim 16, wherein the first computing device is further configured to: control the vehicle according to the control strategy. ### Claim 18 (depends on 16) The system of claim 16, wherein the first computing device is further configured to: detect one or more construction-related street signs; and identify the unusual event based on the one or more construction-related street signs. ### Claim 19 (depends on 16) The system of claim 16, wherein the first computing device is further configured to: transmit, to the second computing device, a natural-language question generated based on the unusual event. ### Claim 20 A non-transitory computer-readable medium having stored thereon program instructions that, if executed by a first computing device, causes the first computing device to perform operations comprising: detecting, based on sensor data from one or more sensors coupled to a vehicle, an unusual event in an environment of the vehicle; determining an action to perform in response to detecting the unusual event; based on determining a confidence level associated with the vehicle performing the action is below a confidence threshold, transmitting, to a second computing device, a request for assistance and the sensor data used to detect the unusual event, wherein the second computing device is positioned remote from the vehicle; receiving a response from the second computing device; and causing the vehicle to perform a control strategy generated based on the response.