# US 2026/0307483 A1 · Visualization of external audio commands

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0307483 A1
- Document ID: US20260307483A1
- Status: Published application
- Kind: A1
- Application number: 19/713,531
- Filed: 2026-06-18
- Published: 2026-10-08
- Earliest claimed: 2024-05-06
- Assignees: Zoox, Inc.
- Applicants: Zoox, Inc.
- Inventors: Joseph WHINNERY; Venkata Subrahmanyam Chandra Sekhar CHEBIYYAM
- Web page: https://patentgrep.com/patent/US20260307483A1

## CPC

- B60K 35/22: Display screens
- G10L 15/26: Speech to text systems (G10L15/08 takes precedence)
- G10L 17/00: Speaker identification or verification techniques
- B60K: ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

This disclosure relates to methods, systems, and techniques for visualizing external audio captured by a microphone of the vehicle. Using the techniques described herein, external audio signals may be interpreted and converted into clear visual indicium that can be understood by individuals associated with the vehicle, such as passengers inside the vehicle or individuals awaiting pickup.

## Claims

### Claim at position 1

1-20. (canceled)

### Claim 21

A system comprising,
one or more processors; and
one or more non-transitory computer-readable media storing instructions executable by the one or more processors, wherein the instructions, when executed, cause the system to perform operations comprising:
recording audio external to a vehicle traversing an environment using a microphone associated with the vehicle, the vehicle an autonomous vehicle;
determining, using one or more sensors associated with the vehicle, that the audio is associated with an authorized entity;
analyzing the audio to determine a command directed towards the vehicle or a passenger of the vehicle, the command comprising an instruction for a passenger of the vehicle to follow; and
causing control of the vehicle based at least in part on the command.

### Claim 22 (depends on 21)

The system of claim 21, the operations comprising identifying a type of the authorized entity using the one or more sensors.

### Claim 23 (depends on 21)

The system of claim 21, the operations comprising determining that the audio is associated with an authorized entity based at least in part on determining a direction of arrival of the audio.

### Claim 24 (depends on 21)

The system of claim 21, the operations comprising:
verifying that the audio originates from the authorized entity.

### Claim 25 (depends on 21)

The system of claim 21, the operations comprising:
determining the command as relating to an area in the environment which a passenger of the vehicle should avoid or proceed to upon exiting the vehicle;
generating a visual representation indicating the area in relation to a current position of the vehicle in the environment; and
displaying the visual representation on a display in the vehicle.

### Claim 26 (depends on 21)

The system of claim 21, the operations comprising:
projecting a light pattern onto a surface in the environment based at least in part on the command, the light pattern indicating a walkable area or a zone to avoid.

### Claim 27

A method comprising:
recording audio using a microphone associated with a vehicle traversing an environment;
determining, from the audio, a command directed towards the vehicle or a passenger of the vehicle, the command comprising an instruction for the vehicle or the passenger of the vehicle to follow; and
causing control of the vehicle based at least in part on the command.

### Claim 28 (depends on 27)

The method of claim 27, comprising:
determining, using one or more sensors associated with the vehicle, that the audio is associated with an authorized entity.

### Claim 29 (depends on 27)

The method of claim 27, comprising:
determining, based at least in part on the audio, that the audio is associated with the vehicle or an occupant of the vehicle.

### Claim 30 (depends on 27)

The method of claim 27, comprising:
determining an object referenced in the command;
detecting the object in the environment using perception data of the vehicle; and
inferring a keep-away area around the detected object based at least in part on at least one of the perception data or map data.

### Claim 31 (depends on 27)

The method of claim 27, comprising:
receiving a signal associated with a remote operator of the vehicle, the signal indicating an update to the command.

### Claim 32 (depends on 27)

The method of claim 27, comprising:
determining a source of the audio in the environment based at least in part on data received from one or more additional sensors associated with the vehicle.

### Claim 33 (depends on 27)

The method of claim 27, comprising:
receiving sensor data from one or more additional sensors associated with the vehicle;
detecting a physical gesture based at least in part on the sensor data; and
determining the command based at least in part on the physical gesture.

### Claim 34

One or more non-transitory computer-readable media storing instructions executable by one or more processors, wherein the instructions, when executed, cause the one or more processors to perform operations comprising:
recording audio using a microphone associated with a vehicle traversing an environment;
determining, based at least in part on the audio, that the audio is associated with the vehicle or an occupant of the vehicle;
determining, based at least in part on the audio, a command directed towards the vehicle or a passenger of the vehicle, the command comprising an instruction for the vehicle or the passenger of the vehicle to follow; and
executing an action based at least in part on the command.

### Claim 35 (depends on 34)

The one or more non-transitory computer-readable media of claim 34, the operations comprising:
determining, based at least in part on one or more sensors of the vehicle and on the command, an area in the environment to avoid.

### Claim 36 (depends on 34)

The one or more non-transitory computer-readable media of claim 34, the operations comprising:
transmitting a notification based on the command to a remote device associated with an individual awaiting the vehicle.

### Claim 37 (depends on 34)

The one or more non-transitory computer-readable media of claim 34, wherein executing the action comprises one or more of:
locking or unlocking a door of the vehicle based at least in part on the command;
transmitting a message associated with the command to a second vehicle or a remote system in communication with a second vehicle;
playing audio via a speaker of the vehicle; or controlling movement of the vehicle based at least in part on the command.

### Claim 38 (depends on 34)

The one or more non-transitory computer-readable media of claim 34, the operations comprising:
determining that a source of the audio is unauthorized; and
outputting a warning based at least in part on determining that the source is unauthorized.

### Claim 39 (depends on 34)

The one or more non-transitory computer-readable media of claim 34, wherein executing the action comprises:
identifying a drop-off location based at least in part on map data and the command; and
controlling the vehicle to drive to the drop-off location.
