# US 10,871,548 B2 · Systems and methods for transient acoustic event detection, classification, and localization

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,871,548 B2
- Document ID: US10871548B2
- Status: Granted
- Kind: B2
- Application number: 15/368,632
- Filed: 2016-12-04
- Published: 2017-11-16
- Granted: 2020-12-22
- Earliest claimed: 2015-12-04
- Assignees: Fazecast, Inc.
- Applicants: Fazecast, Inc.
- Inventors: Peter Volgyesi; Janos Sallai; Akos Ledeczi; Gyorgy Balogh
- Web page: https://patentgrep.com/patent/US10871548B2

## CPC

- G01S 5/22: Position of source determined by co-ordinating a plurality of position lines defined by path-difference measurements (G01S5/28 takes precedence)
- G01S 5/0036: {of measured values, i.e. measurement on mobile and position calculation on base station}
- G01S 5/28: by co-ordinating position lines of different shape, e.g. hyperbolic, circular, elliptical or radial
- G01S 5/18: using ultrasonic, sonic or infrasonic waves
- G01S 19/13: Receivers
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES

## Abstract

A system is described that comprises a plurality of sensor nodes and at least one remote server, wherein each sensor node of the plurality of sensor nodes and the at least one remote server are communicatively coupled, wherein the plurality of sensor nodes receive at least one acoustic signal, process the at least one acoustic signal to detect one or more transient events, classify the one or more transient events as an event type, and determine geometry information and timing information of the one or more transient events. The system comprises at least one of the plurality of sensor nodes and the at least one remote server identifying the source of the one or more transient events.

## Claims

### Claim 1

A system comprising,
a plurality of sensor nodes and at least one remote server, wherein each sensor node of the plurality of sensor nodes and the at least one remote server are communicatively coupled, wherein the plurality of sensor nodes receive at least one acoustic signal, process the at least one acoustic signal to detect one or more transient events, classify the one or more transient events as an event type, and determine geometry information and timing information of the one or more transient events, wherein the detecting the one or more transient event includes applying a detection algorithm to the digital signal, wherein each sensor nodes of the plurality of sensor nodes comprises a GPS unit and at least four acoustic channels for receiving the at least one acoustic signal, wherein each sensor node of the plurality of sensor nodes continuously samples the at least four acoustic channels and converts the at least one acoustic signal into a digital signal, wherein the digital signal comprises a 4-channel digital signal;
at least one of the plurality of sensor nodes and the at least one remote server identifying the source of the one or more transient events, wherein the identifying the source of the one or more transient events includes applying a convolutional neural network analysis to the digital signal to classify the at least one acoustic signal as an event type, wherein input of the convolutional neural network analysis includes a timestamp of the one or more transient events, wherein the input includes a Short-Term-Fourier Transform (STFT) of one or more of the at least one acoustic signal and a high-pass filtered version of the at least one acoustic signal near a transient trigger.

### Claim 2 (depends on 1)

The system of claim 1, wherein the communicative coupling comprises at least one of a wired and wireless network.

### Claim 3 (depends on 1)

The system of claim 1, wherein the plurality of sensor nodes exchange at least one of geometry information, timing information, information of the one or more transient events, and information of the event type.

### Claim 4 (depends on 1)

The system of claim 1, wherein the plurality of sensor nodes provide the at least one remote server at least one of geometry information, timing information, information of the one or more transient events, and information of the event type.

### Claim 5 (depends on 1)

The system of claim 1, the applying the detection algorithm including applying a Short Time Fourier Transform to the digital signal using a fixed frequency resolution and a time overlap.

### Claim 6 (depends on 1)

The system of claim 1, the applying the detection algorithm including computing a change in spectral magnitudes in adjacent timeslots of selected frequency bins using the L2-norm, wherein the resulting L2 distance values comprise spectral flux measurements.

### Claim 7 (depends on 6)

The system of claim 6, the applying the detection algorithm including identifying a potential transient event of interest when the spectral flux measurements rise above an adaptive threshold level.

### Claim 8 (depends on 7)

The system of claim 7, computing the adaptive threshold level by applying an exponential filter with an absolute minimum value and a rising edge override to the spectral flux measurements.

### Claim 9 (depends on 1)

The system of claim 1, wherein the STFT uses overlapping windows and a frequency resolution, wherein the frequency resolution is greater than 20 Hz.

### Claim 10 (depends on 9)

The system of claim 9, wherein the STFT provides an input vector to the CNN, wherein the input vector comprises a two-dimensional data structure including spectral magnitude information, wherein the two-dimensional data structure preserves time-varying dynamics of the signal.

### Claim 11 (depends on 10)

The system of claim 10, wherein the CNN employs a stack of convolutional layers at its input connected by non-linear activation functions, pooling layers, and drop-out layers, wherein convolution layers comprise individual filters that generate 1D outputs.

### Claim 12 (depends on 11)

The system of claim 11, wherein the CNN merges the 1D outputs of the filters to provide new 2D inputs for subsequent convolutional layers.

### Claim 13 (depends on 12)

The system of claim 12, wherein the pooling layers are used to shrink and decimate the signals in the time domain after each convolutional filtering.

### Claim 14 (depends on 13)

The system of claim 13, wherein after convolution and pooling, the CNN uses traditional fully-connected layers interspersed with rectifier linear units (ReLU), the activations of which are computed by matrix multiplications with added bias offsets.

### Claim 15 (depends on 14)

The system of claim 14, wherein the size of the fully-connected layers decreases progressively such that the final layer includes exactly as many output activations as the number of event types the system targets, wherein event types include gunshot, firework, and car crash events.

### Claim 16 (depends on 15)

The system of claim 15, wherein the weights and biases used in the CNN classification network are calculated through a supervised learning process with a large set of labeled example data, wherein the identifying the source of the one or more transient events includes accessing the weights and biases stored locally in the plurality of sensor nodes.

### Claim 17 (depends on 15)

The system of claim 15, wherein the output of the CNN is a probability vector that assigns the one or more acoustic events a likelihood of representing an event type.

### Claim 18 (depends on 1)

The system of claim 1, wherein the geometry information comprises Angle of Arrival (AoA) of the one or more transient events.

### Claim 19 (depends on 18)

The system of claim 18, the determining the geometry information comprising estimating the AoA by cross-correlating pairs of channels of the plurality of acoustic channels on a sensor node of the plurality of sensor nodes.

### Claim 20 (depends on 18)

The system of claim 18, wherein the timing information includes Time of Arrival (ToA) of the one or more transient events.

### Claim 21 (depends on 20)

The system of claim 20, the determining the timing information comprising estimating the ToA on a precise absolute time scale by timestamping the maximum of a cross-correlation function within the at least one acoustic signal using a Pulse Per Second (PPS) pin of the GPS unit in a sensor node of the plurality of sensor nodes.

### Claim 22 (depends on 20)

The system of claim 20, the identifying the source of the one or more transient events including designating a location for each sensor device of the plurality of sensor devices as (xi,yi).

### Claim 23 (depends on 22)

The system of claim 22, the identifying the source of the one or more transient events including designating a ToA of the at least one acoustic signal with respect to each sensor device of the plurality of sensor devices as ti.

### Claim 24 (depends on 23)

The system of claim 23, the identifying the source of the one or more transient events including describing a conical surface for each sensor node of the plurality of sensor nodes, the describing including extending a cone from location (xi, yi, ti), wherein ti comprises a third dimension perpendicular to the (xi, yi) plane.

### Claim 25 (depends on 24)

The system of claim 24, wherein a cone angle of the conical surface for each sensor node of the plurality of sensor nodes is calculated as tan+1c, wherein c comprises the speed of sound.

### Claim 26 (depends on 25)

The system of claim 25, the identifying the source of the one or more transient events including providing a Hough accumulator matrix, wherein the Hough accumulator matrix provides two dimensional locations (x, y) spanning a geographic area.

### Claim 27 (depends on 26)

The system of claim 26, wherein the Hough accumulator matrix provides a third dimension t, wherein the Hough accumulator matrix comprises matrix bins (x, y, t).

### Claim 28 (depends on 27)

The system of claim 27, the identifying the source of the one or more transient events including incrementing a bin of the accumulator matrix when a conical surface of any sensor node of the plurality of sensor nodes intersects with a bin.

### Claim 29 (depends on 28)

The system of claim 28, the identifying the source of the one or more transient events including incrementing a bin only when the bin in is in a direction of an AoA with respect to any sensor node of the plurality of sensor nodes.

### Claim 30 (depends on 28)

The system of claim 28, the identifying the source of the one or more transient events including invoking a peak detector on the accumulator matrix to extract parameters of the local maxima equal to or above a threshold of 3 or more, wherein (x1,yi) coordinates of the maxima are potential source positions of the one or more transient events, and tj values are the times of the one or more transient events, where the subscript j indexes the one or more transient events.

### Claim 31 (depends on 30)

The system of claim 30, the identifying the source of the one or more transient events including merging the potential source positions based on their locality in order to overcome quantization errors in the accumulator matrix resulting in different local maxima corresponding to a same event of the one or more transient events.

### Claim 32 (depends on 31)

The system of claim 31, the identifying the source of the one or more transient events including applying a nonlinear optimization procedure to solve the multilateration problem for the merged potential source positions and corresponding ToAs, resulting in a fine-grained set of source positions and emission times for the one or more transient events.