# US 2019/0376840 A1 · Anomalous sound detection apparatus, degree-of-anomaly calculation apparatus, anomalous sound generation apparatus, anomalous sound detection training apparatus, anomalous signal detection apparatus, anomalous signal detection training apparatus, and methods and programs therefor

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0376840 A1
- Document ID: US20190376840A1
- Status: Published application
- Kind: A1
- Application number: 16/485,334
- Filed: 2017-09-14
- Published: 2019-12-12
- Earliest claimed: 2017-02-15
- Assignees: Nippon Telegraph and Telephone Corporation
- Applicants: Nippon Telegraph and Telephone Corporation
- Inventors: Yuma KOIZUMI; Shoichiro SAITO; Hisashi UEMATSU
- Web page: https://patentgrep.com/patent/US20190376840A1

## CPC

- G01H 17/00: Measuring mechanical vibrations or ultrasonic, sonic or infrasonic waves, not provided for in the other groups of this subclass
- G01N 29/4427: {with stored values, e.g. threshold values}
- G01N 29/14: using acoustic emission techniques {(echo of particles G01N29/046; measuring mechanical vibrations or acoustic waves in solids in general G01H1/00)}
- G01N 29/4445: {Classification of defects}
- G01N 29/4472: {Mathematical theories or simulation}
- G01N 29/46: by spectral analysis, e.g. Fourier analysis {or wavelet analysis (spectral signal processing per se G06F17/14)}
- G01N 29/52: using inversion methods other that spectral analysis, e.g. conjugated gradient inversion
- G06N 3/045: Combinations of networks
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/047: Probabilistic or stochastic networks
- G06N 3/088: Non-supervised learning, e.g. competitive learning
- G06N 3/0895: Weakly supervised learning, e.g. semi-supervised or self-supervised learning
- G06N 3/09: Supervised learning
- G10L 25/03: characterised by the type of extracted parameters
- G10L 25/51: for comparison or discrimination
- G01N 2291/0258: Structural degradation, e.g. fatigue of composites, ageing of oils
- G01H: MEASUREMENT OF MECHANICAL VIBRATIONS OR ULTRASONIC, SONIC OR INFRASONIC WAVES
- G01N: INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES (measuring or testing apparatus or processes other than immunoassay, involving enzymes or microorganisms C12M, C12Q)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

To provide an anomalous sound detection training technique by which a feature amount extraction function for detecting anomalous sound can be generated irrespective of whether training data for anomalous signals is available or not. An anomalous sound detection training apparatus includes: a first function updating unit 3 that updates a feature amount extraction function and an feature amount inverse transformation function, which are input, based on an optimization index of a variational autoencoder; an acoustic feature extraction unit 4 that extracts an acoustic feature of normal sound based on training data for normal sound; a normal sound model updating unit 5 that updates a normal sound model by using the acoustic feature that is extracted; a threshold updating unit 6 that obtains a threshold φρ corresponding to a false positive rate ρ, which has a predetermined value, by using the training data for normal sound and the feature amount extraction function that is input; and a second function updating unit 8 that updates the feature amount extraction function that is updated, based on a Neyman-Pearson-type optimization index defined by the threshold φρ that is obtained, and repeatedly performs processing of each of the above-mentioned units.

## Claims

### Claim 1

An anomalous sound detection apparatus that detects whether or not input sound, the input sound being input, is anomalous sound, the anomalous sound detection apparatus comprising:
an acoustic feature extraction unit that extracts an acoustic feature of the input sound by using a feature amount extraction function;
a degree-of-anomaly calculation unit that calculates a degree of anomaly of the input sound by using the acoustic feature that is extracted; and
a determination unit that determines whether or not the input sound is anomalous sound based on the degree of anomaly that is obtained and a threshold, wherein
the feature amount extraction function is based on probability distribution modeling sound which can include normal sound and anomalous sound, probability distribution modeling normal sound, and probability distribution modeling anomalous sound that is input, and
the threshold is set by using degrees of anomaly obtained from the normal sound.

### Claim 2 (depends on 1)

The anomalous sound detection apparatus according to claim 1, wherein
the feature amount extraction function is set so that a degree of anomaly calculated from an acoustic feature of normal sound, the acoustic feature of normal sound being obtained by using the feature amount extraction function, is smaller than the threshold and a degree of anomaly calculated from an acoustic feature of anomalous sound, the anomalous sound being input, the acoustic feature of the anomalous sound being obtained by using the feature amount extraction function, is greater than the threshold.

### Claim 3 (depends on 2)

The anomalous sound detection apparatus according to claim 2, wherein
the feature amount extraction function is a function that is generated through first update based on an optimization index of a variational autoencoder and second update based on an index that is defined based on the acoustic feature of normal sound, the acoustic feature of anomalous sound, and the threshold.

### Claim 4 (depends on 3)

The anomalous sound detection apparatus according to claim 3, wherein
the index is a Neyman-Pearson-type optimization index.

### Claim 5 (depends on 1)

The anomalous sound detection apparatus according to claim 1, wherein
the probability distribution modeling normal sound is probability distribution obtained by removing the probability distribution modeling normal sound from the probability distribution modeling sound which can include normal sound and anomalous sound.

### Claim 6 (depends on 1)

The anomalous sound detection apparatus according to claim 1, wherein
the anomalous sound, the anomalous sound being input, is generated by using the probability distribution modeling sound which can include normal sound and anomalous sound, a feature amount inverse transformation function that is an inverse function of the feature amount extraction function, and the threshold.

### Claim 7 (depends on 6)

The anomalous sound detection apparatus according to claim 6, wherein
the anomalous sound, the anomalous sound being input, is generated by generating an acoustic feature fks following Formula (23) below:
fks˜p(F(x))−p(F(x)|z=0) (23)
when p(F(x)) is set as the probability distribution modeling sound which can include normal sound and anomalous sound and p(F(x)|z=0) is set as the probability distribution modeling normal sound.

### Claim 8 (depends on 1, 2, 3, 4, 5, 6, 7)

The anomalous sound detection apparatus according to claims 1 to 7, wherein
the threshold is an N-th degree of anomaly from a top among the degrees of anomaly that are obtained from the normal sound and are sorted in a descending order, when N is set as a predetermined positive integer.

### Claim 9

A degree-of-anomaly calculation apparatus that calculates a degree of anomaly of input sound, the input sound being input, so as to detect whether or not the input sound is anomalous sound, the degree-of-anomaly calculation apparatus comprising:
an acoustic feature extraction unit that extracts an acoustic feature of the input sound by using a feature amount extraction function; and
a degree-of-anomaly calculation unit that calculates a degree of anomaly of the input sound by using the acoustic feature that is extracted, wherein
the feature amount extraction function is based on probability distribution modeling sound which can include normal sound and anomalous sound, probability distribution modeling normal sound, and probability distribution modeling anomalous sound that is input, and
the threshold is set by using degrees of anomaly obtained from the normal sound.

### Claim 10

An anomalous sound generation apparatus that generates anomalous sound based on sound which can include normal sound and anomalous sound, the anomalous sound generation apparatus comprising:
an anomalous sound generation unit that generates anomalous sound by using probability distribution modeling the sound which can include normal sound and anomalous sound, the feature amount inverse transformation function that is an inverse function of the feature amount extraction function, and a threshold, wherein
the feature amount inverse transformation function is an inverse transformation function of a feature amount transformation function that is based on the probability distribution modeling the sound which can include normal sound and anomalous sound, probability distribution modeling normal sound, and probability distribution modeling anomalous sound that is input, and
the threshold is set by using degrees of anomaly obtained from the normal sound.

### Claim 11 (depends on 10)

The anomalous sound generation apparatus according to claim 10, wherein
the anomalous sound generation unit generates the anomalous sound such that the anomalous sound generation unit generates a value obtained by approximating an acoustic feature following the probability distribution modeling the sound which can include normal sound and anomalous sound, calculates a degree of anomaly of the value that is generated, and calculates an output value obtained by inputting the value that is generated to the feature amount inverse transformation function when the degree of anomaly, the degree of anomaly being calculated, is greater than the threshold.

### Claim 12

An anomalous sound detection training apparatus comprising:
a first function updating unit that updates a feature amount extraction function and an feature amount inverse transformation function, the feature amount extraction function and the feature amount inverse transformation function being input, based on an optimization index of a variational autoencoder;
an acoustic feature extraction unit that extracts an acoustic feature of normal sound based on training data for normal sound by using the feature amount extraction function that is input;
a normal sound model updating unit that updates a normal sound model by using the acoustic feature that is extracted;
a threshold updating unit that obtains a threshold φρ corresponding to a false positive rate ρ, the false positive rate p having a predetermined value, by using the training data for normal sound and the feature amount extraction function that is input; and
a second function updating unit that updates the feature amount extraction function that is updated, based on a Neyman-Pearson-type optimization index defined by the threshold φρ that is obtained, by using the acoustic feature of normal sound that is extracted and an acoustic feature of anomalous sound that is input, wherein
processing of the first function updating unit, processing of the acoustic feature extraction unit, processing of the normal sound model updating unit, and processing of the second function updating unit are repeatedly performed by using the feature amount extraction function, the feature amount extraction function being updated by the second function updating unit, as an input.

### Claim 13 (depends on 12)

The anomalous sound detection training apparatus according to claim 12, further comprising:
an anomalous sound sampling unit that generates the acoustic feature of anomalous sound.

### Claim 14 (depends on 13)

The anomalous sound detection training apparatus according to claim 13, wherein
the anomalous sound sampling unit generates the acoustic feature of anomalous sound by generating an acoustic feature fks of anomalous sound following Formula (23) below:
fks˜p(F(x))−p(F(x)|z=0) (23)
when p(F(x)|z≠0) is set as a probability distribution followed by anomalous sound, p(F(x)) is set as a probability distribution followed by sound which can include normal sound and anomalous sound, and p(F(x)|z=0) is set as a probability distribution followed by normal sound.

### Claim 15

An anomalous signal detection apparatus that detects whether or not an input signal, the input signal being input, is an anomalous signal, the anomalous signal detection apparatus comprising:
a feature amount extraction unit that extracts a feature amount of the input signal by using a feature amount extraction function;
a degree-of-anomaly calculation unit that calculates a degree of anomaly of the input signal by using the feature amount that is extracted; and
a determination unit that determines whether or not the input signal is an anomalous signal based on the degree of anomaly that is obtained and a threshold, wherein
the feature amount extraction function is based on probability distribution modeling signals which can include a normal signal and an anomalous signal, probability distribution modeling normal signals, and probability distribution modeling anomalous signals that are input, and
the threshold is set by using degrees of anomaly obtained from the normal signal.

### Claim 16

An anomalous signal detection training apparatus comprising:
a first function updating unit that updates a feature amount extraction function and an feature amount inverse transformation function, the feature amount extraction function and the feature amount inverse transformation function being input, based on an optimization index of a variational autoencoder;
a feature amount extraction unit that extracts a feature amount of a normal signal based on training data for normal signals by using the feature amount extraction function that is input;
a normal signal model updating unit that updates a normal signal model by using the feature amount that is extracted;
a threshold updating unit that obtains a threshold φρ corresponding to a false positive rate ρ, the false positive rate p having a predetermined value, by using the training data for normal signals and the feature amount extraction function that is input; and
a second function updating unit that updates the feature amount extraction function that is updated, based on a Neyman-Pearson-type optimization index defined by the threshold φρ that is obtained, by using the feature amount of a normal signal that is extracted and a feature amount of an anomalous signal that is input, wherein
processing of the first function updating unit, processing of the feature amount extraction unit, processing of the normal signal model updating unit, and processing of the second function updating unit are repeatedly performed by using the feature amount extraction function, the feature amount extraction function being updated by the second function updating unit, as an input.

### Claim 17

An anomalous sound detection method for detecting whether or not input sound, the input sound being input, is anomalous sound, the anomalous sound detection method comprising:
an acoustic feature extraction step in which an acoustic feature extraction unit extracts an acoustic feature of the input sound by using a feature amount extraction function;
a degree-of-anomaly calculation step in which a degree-of-anomaly calculation unit calculates a degree of anomaly of the input sound by using the acoustic feature that is extracted; and
a determination step in which a determination unit determines whether or not the input sound is anomalous sound based on the degree of anomaly that is obtained and a threshold, wherein
the feature amount extraction function is based on probability distribution modeling sound which can include normal sound and anomalous sound, probability distribution modeling normal sound, and probability distribution modeling anomalous sound that is input, and
the threshold is set by using degrees of anomaly obtained from the normal sound.

### Claim 18

An anomalous sound detection training method comprising:
a first function updating step in which a first function updating unit updates a feature amount extraction function and an feature amount inverse transformation function, the feature amount extraction function and the feature amount inverse transformation function being input, based on an optimization index of a variational autoencoder;
an acoustic feature extraction step in which an acoustic feature extraction unit extracts an acoustic feature of normal sound based on training data for normal sound by using the feature amount extraction function that is input;
a normal sound model updating step in which a normal sound model updating unit updates a normal sound model by using the acoustic feature that is extracted;
a threshold updating step in which a threshold updating unit obtains a threshold φρ corresponding to a false positive rate ρ, the false positive rate p having a predetermined value, by using the training data for normal sound and the feature amount extraction function that is input; and
a second function updating step in which a second function updating unit updates at least one of the feature amount extraction function that is updated and the feature amount inverse transformation function that is updated, based on a Neyman-Pearson-type optimization index defined by the threshold φρ that is obtained, by using the acoustic feature of normal sound that is extracted and an acoustic feature of anomalous sound that is input, wherein
processing of the first function updating step, processing of the acoustic feature extraction step, processing of the normal sound model updating step, and processing of the second function updating step are repeatedly performed by using the feature amount extraction function and the feature amount inverse transformation function, the feature amount extraction function and the feature amount inverse transformation function being updated by the first function updating step, as inputs.

### Claim 19

An anomalous signal detection method for detecting whether or not an input signal, the input signal being input, is an anomalous signal, the anomalous signal detection method comprising:
a feature amount extraction step in which a feature amount extraction unit extracts a feature amount of the input signal by using a feature amount extraction function;
a degree-of-anomaly calculation step in which a degree-of-anomaly calculation unit calculates a degree of anomaly of the input signal by using the feature amount that is extracted; and
a determination step in which a determination unit determines whether or not the input signal is an anomalous signal based on the degree of anomaly that is obtained and a threshold, wherein
the feature amount extraction function is based on probability distribution modeling signals which can include a normal signal and an anomalous signal, probability distribution modeling normal signals, and probability distribution modeling anomalous signals that are input, and
the threshold is set by using degrees of anomaly obtained from the normal signal.

### Claim 20 (depends on 1, 9, 10, 12, 15)

A program for making a computer function as each unit of any of the apparatuses according to claims 1, 9, 10, 12, 15, and 16.