US 11,322,019 B2Granted

Emergency vehicle detection

AssigneeZoox, Inc.

InventorsNam Gook Cho, Subasingha Shaminda Subasingha, Jonathan Tyler Dowdall, Venkata Subrahmanyam Chandra Sekhar Chebiyyam

Life of this patent

Estimating the term…

patentgrep

Loading patent…

Abstract

Techniques for determining a direction of arrival of an emergency are discussed. A plurality of audio sensors of a vehicle can receive audio data associated with the vehicle. An audio sensor pair can be selected from the plurality of audio sensors to generate audio data representing sound in an environment of the vehicle. An angular spectrum associated with the audio sensor pair can be determined based on the audio data. A feature associated with the audio data can be determined based on the angular spectrum and/or the audio data itself. A direction of arrival (DoA) value associated with the sound may be determined based on the feature using a machine learned model. An emergency sound (e.g., a siren) can be detected in the audio data and a direction associated with the emergency relative to the vehicle can be determined based on the feature and the DoA value.

Claims20

Dependent claims

A system comprising:

one or more processors;
and
one or more computer-readable media storing instructions executable by the one or more processors, wherein the instructions, when executed, cause the system to perform operations comprising:
receiving audio data from a pair of audio sensors associated with a vehicle;
determining, based at least in part on a portion of the audio data, angular spectrum data;
determining, based at least in part on the angular spectrum data, at least one of a distribution of the angular spectrum data, a peak value of the angular spectrum data, or an energy value of the audio data;
inputting into a machine learned model, the at least one of the distribution of the angular spectrum data, the peak value of the angular spectrum data, or the energy value of the audio data;
determining, by the machine learned model and based at least in part on the at least one of the distribution of the angular spectrum data, the peak value of the angular spectrum data, or the energy value associated with the audio data, a direction of arrival (DoA) value associated with a time difference of arrival (TDoA) value of a peak of the angular spectrum data;
receiving, from the machine learned model, the DoA value;
determining, based at least in part on the audio data, an occurrence of sound associated with an emergency vehicle;
and
determining, based at least in part on the DoA value, a direction of the emergency vehicle relative to the vehicle.

9 more elements

A method comprising:

receiving audio data from a first audio sensor and a second audio sensor associated with a vehicle;
determining at least one of a distribution of angular spectrum data, a peak value of the angular spectrum data, or an energy value associated with the audio data;
inputting into a machine learned model, the at least one of the distribution of the angular spectrum data, the peak value of the angular spectrum data, or the energy value of the audio data;
determining, by the machine learned model and based at least in part on the at least one of the distribution of the angular spectrum data, the peak value of the angular spectrum data, or the energy value associated with the audio data, a direction of arrival (DoA) value associated with a time difference of arrival (TDoA) value of a peak of the angular spectrum data;
receiving, from the machine learned model, the DoA value;
determining, based at least in part on the audio data, an occurrence of an audio event;
determining, based at least in part on the DoA value, a direction of the audio event relative to the vehicle;
and
controlling the vehicle based at least in part on the direction of the audio event relative to the vehicle.

6 more elements

A non-transitory computer-readable medium storing instructions that, when executed, cause one or more processors to perform operations comprising:

receiving audio data from a first audio sensor and a second audio sensor associated with a vehicle;
determining at least one of a distribution of angular spectrum data, a peak value of the angular spectrum data, or an energy value associated with the audio data;
inputting into a machine learned model, the at least one of the distribution of the angular spectrum data, the peak value of the angular spectrum data, or the energy value of the audio data;
determining, by the machine learned model and based at least in part on the at least one of the distribution of the angular spectrum data, the peak value of the angular spectrum data, or the energy value associated with the audio data, a direction of arrival (DoA) value associated with a time difference of arrival (TDoA) value of a peak of the angular spectrum data;
receiving, from the machine learned model, the DoA value;
determining, based at least in part on the audio data, an occurrence of an audio event;
determining, based at least in part on the DoA value, a direction of the audio event relative to the vehicle;
and
controlling the vehicle based at least in part on the direction of the audio event relative to the vehicle.

6 more elements

Dependencies are inferred from the claim text.

Facts

Application: 16/661,898Filed 23 Oct 2019
Published: US 2021/0125494 A129 Apr 2021
Granted: 3 May 2022
Term ends: Not estimatedThe source records don’t support an estimate.

Classification10 codes

G08G 1/04
Detecting movement of traffic to be counted or controlled: using optical or ultrasonic detectors
G01S 3/8032
Using amplitude comparison of signals derived from receiving transducers or transducer systems having differently-oriented directivity…
G01S 3/8083
Using transducers spaced apart and measuring phase or time difference between signals therefrom, i.e. path-difference systems: determining…
G06F 18/21
Analysing: design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation