# US 2022/0284919 A1 · Detection and classification of siren signals and localization of siren signal sources

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0284919 A1
- Document ID: US20220284919A1
- Status: Published application
- Kind: A1
- Application number: 17/713,008
- Filed: 2022-04-04
- Published: 2022-09-08
- Earliest claimed: 2020-01-24
- Assignees: Motional AD LLC
- Applicants: Motional AD LLC
- Inventors: Jugal Buddhadev; Rajesh K. Venkateswaran; Alok Sharma; Yunpeng Xu; Michael Lee; Paul Schmitt; Matthew Andromalos
- Web page: https://patentgrep.com/patent/US20220284919A1

## CPC

- G10L 25/51: for comparison or discrimination
- G01S 3/8006: {Multi-channel systems specially adapted for direction-finding, i.e. having a single aerial system capable of giving simultaneous indications of the directions of different signals}
- G01S 3/8083: {determining direction of source}
- G01S 5/22: Position of source determined by co-ordinating a plurality of position lines defined by path-difference measurements (G01S5/28 takes precedence)
- G01S 11/14: using ultrasonic, sonic or infrasonic waves
- G10L 25/18: the extracted parameters being spectral information of each sub-band
- G10L 25/24: the extracted parameters being the cepstrum
- G10L 25/30: using neural networks
- H04R 1/406: {microphones}
- H04R 3/005: {for combining the signals of two or more microphones (specially adapted for hearing aids H04R25/407)}
- G01S 2205/01: specially adapted for specific applications
- H04R 2430/21: Direction finding using differential microphone array [DMA]
- H04R 2499/13: Acoustic transducers and sound field adaptation in vehicles
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K)

## Abstract

In an embodiment, a method comprises: capturing, by one or more microphone arrays of a vehicle, sound signals in an environment; extracting frequency spectrum features from the sound signals; predicting, using an acoustic scene classifier and the frequency spectrum features, one or more siren signal classifications; converting the one or more siren signal classifications into one or more siren signal event detections; computing time delay of arrival estimates for the one or more detected siren signals; estimating one or more bearing angles to one or more sources of the one or more detected siren signals using the time delay of arrival estimates and a known geometry of the microphone array; and tracking, using a Bayesian filter, the one or more bearing angles. If a siren is detected, actions are performed by the vehicle depending on the location of the emergency vehicle and whether the emergency vehicle is active or inactive.

## Claims

### Claim 1

A method comprising:
capturing, by one or more microphone arrays of a vehicle, sound signals in an environment;
extracting, using one or more processors, frequency spectrum features from the sound signals;
predicting, using an acoustic scene classifier and the frequency spectrum features, one or more siren signal classifications, wherein the acoustic scene classifier predicts labels that indicate the presence of one or more of a plurality of different types of siren signals;
converting, using the one or more processors, the one or more siren signal classifications into one or more siren signal event detections;
computing time delay of arrival estimates for the one or more detected siren signals;
estimating, using the one or more processors, one or more bearing angles to one or more sources of the one or more detected siren signals using the time delay of arrival estimates and a known geometry of the microphone array; and
tracking, using a Bayesian filter, the one or more bearing angles.

### Claim 2 (depends on 1)

The method of claim 1, wherein the time delay of arrival estimates are computed using a maximum likelihood criterion obtained by implementing a generalized cross correlation method.

### Claim 3 (depends on 1)

The method of claim 1, further comprising estimating one or more ranges of the one or more siren signal sources by applying triangularization to the one or more bearing angles.

### Claim 4 (depends on 1)

The method of claim 1, wherein transforming sound signals into frequency spectrum features includes generating one of a spectrogram, mel-spectrogram or mel-frequency cepstral coefficients (MFCC).

### Claim 5 (depends on 1)

The method of claim 1, wherein the acoustic scene classifier is implemented at least in part using a convolutional neural network (CNN).

### Claim 6 (depends on 1)

The method of claim 1, wherein the Bayesian filter is one of a Kalman filter, extended Kalman filter (EKF), unscented Kalman filter or particle filter.

### Claim 7 (depends on 1)

The method of claim 1, wherein predicting, using an acoustic scene classifier and the frequency spectrum features, one or more siren signal classifications, further comprises continuously predicting labels indicating the presence or absence of the one or more types of siren signals.

### Claim 8 (depends on 1)

The method of claim 1, wherein the one or more bearing angles are estimated by using a spatio-temporal difference of the one or more detected siren signals at each microphone pair in the microphone array.

### Claim 9

(canceled)

### Claim 10 (depends on 1)

The method of claim 1, wherein the different types of siren signals include one or more of wailing, yelp, hi-lo, rumbler, chirp, pulsar, localizer, and mechanical wail siren signals.

### Claim 11 (depends on 1)

The method of claim 1, wherein the bearing angles are used to triangulate the location of the sound source.

### Claim 12 (depends on 11)

The method of claim 11, wherein the sound source is associated with an emergency vehicle and the method further comprises causing, using the one or more processors and the location of the sound source, to operate the vehicle in accordance with one or more rules associated with emergency vehicles.

### Claim 13 (depends on 11)

The method of claim 11, further comprising determining whether the emergency vehicle is active or inactive, and if active, operating the vehicle in accordance with a first set of rules associated with active emergency vehicles, or if inactive, operating the vehicle in accordance with a second set of rules associated with an inactive emergency vehicles.

### Claim 14 (depends on 13)

The method of claim 13, wherein if the emergency vehicle is active and nearby and the vehicle has crossed a stop line at an intersection, causing the vehicle to initiate a comfort stop, or if the emergency vehicle is active and far away and the vehicle has crossed the stop line, causing the vehicle to traverse across the intersection and then initiate a comfort stop.

### Claim 15 (depends on 13)

The method of claim 13, wherein the emergency vehicle is active and the vehicle is within a left lane, and a right lane is open and available, causing the AV to merge into the right lane.

### Claim 16 (depends on 13)

The method of claim 13, wherein the emergency vehicle is active and the AV is within the rightmost lane, causing the AV to bias to the right, if possible but not cross the lane boundary, and then initiate a comfort stop and remains stopped until the following conditions are met:
1) the emergency vehicle is traveling away from the AV with a range rate that is greater than a specified speed for greater than a specified time; and
2) the emergency vehicle range is greater than a specified distance, or the emergency vehicle is no longer detected for greater than a specified time, and if the conditions are met causing the AV to resume its route towards a goal point.

### Claim 17 (depends on 13)

The method of claim 13, wherein the emergency vehicle is inactive, the method further comprising:
determining whether the AV is on a same road as the emergency vehicle;
determining whether the AV is on the same side of the road or an opposite side of the road as the emergency vehicle;
if on the same road as the emergency vehicle,
determining whether the vehicle is in in front or of behind the emergency vehicle; and
causing the vehicle to travel a trajectory to avoid collision with the emergency vehicle based on whether the vehicle is on the same road or a different road, the same side of road or the opposite side, and if on the same side of the road whether the vehicle is in front of or behind the emergency vehicle.

### Claim 18 (depends on 13)

The method of claim 13, wherein the emergency vehicle is inactive, further comprising:
determining that the vehicle is on a road with single lane, and the emergency vehicle is located fully on a shoulder area and fully or partly within a lane in which the vehicle is traveling; and
causing the vehicle to initiate a comfort stop is initiated; and
causing the vehicle to remain stopped until the emergency vehicle is located fully on the shoulder area, and then to proceed with a maximum speed limit or switch to active and travel away from the emergency vehicle.

### Claim 19 (depends on 1)

The method of claim 1, wherein the frequency spectrum features comprises an oscillation frequency of the sound signals, and the acoustic scene classifier predicts labels that indicate the presence of one or more of a plurality of different types of siren signals based, at least in part, on the oscillation frequency.

### Claim 20

A vehicle comprising:
one or more microphone arrays;
one or more processors;
memory storing instructions that when executed by the one or more processors, cause the one or more processors to perform operations comprising:
capturing, by one or more microphone arrays of a vehicle, sound signals in an environment;
extracting, using one or more processors, frequency spectrum features from the sound signals;
predicting, using an acoustic scene classifier and the frequency spectrum features, one or more siren signal classifications, wherein the acoustic scene classifier predicts labels that indicate the presence of one or more of a plurality of different types of siren signals;
converting, using the one or more processors, the one or more siren signal classifications into one or more siren signal event detections;
computing time delay of arrival estimates for the one or more detected siren signals;
estimating, using the one or more processors, one or more bearing angles to one or more sources of the one or more detected siren signals using the time delay of arrival estimates and a known geometry of the microphone array; and
tracking, using a Bayesian filter, the one or more bearing angles.

### Claim 21

One or more non-transitory storage media storing instructions which, when executed by one or more computing devices, cause one or more processors to perform operations comprising:
capturing, by one or more microphone arrays of a vehicle, sound signals in an environment;
extracting, using one or more processors, frequency spectrum features from the sound signals;
predicting, using an acoustic scene classifier and the frequency spectrum features, one or more siren signal classifications, wherein the acoustic scene classifier predicts labels that indicate the presence of one or more of a plurality of different types of siren signals;
converting, using the one or more processors, the one or more siren signal classifications into one or more siren signal event detections;
computing time delay of arrival estimates for the one or more detected siren signals;
estimating, using the one or more processors, one or more bearing angles to one or more sources of the one or more detected siren signals using the time delay of arrival estimates and a known geometry of the microphone array; and
tracking, using a Bayesian filter, the one or more bearing angles.
