# US 2024/0201009 A1 · Abnormal sound detection in a mechanized environment

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0201009 A1
- Document ID: US20240201009A1
- Status: Published application
- Kind: A1
- Application number: 18/539,086
- Filed: 2023-12-13
- Published: 2024-06-20
- Earliest claimed: 2022-12-16
- Assignees: Micron Technology, Inc.
- Applicants: Micron Technology, Inc.
- Inventors: Ming Chen
- Web page: https://patentgrep.com/patent/US20240201009A1

## CPC

- G01H 17/00: Measuring mechanical vibrations or ultrasonic, sonic or infrasonic waves, not provided for in the other groups of this subclass
- G06F 16/433: {using audio data}
- G06F 16/60: of audio data
- G07C 3/00: Registering or indicating the condition or the working of machines or other apparatus, other than vehicles
- G01N 29/14: using acoustic emission techniques {(echo of particles G01N29/046; measuring mechanical vibrations or acoustic waves in solids in general G01H1/00)}
- G01N 29/4427: {with stored values, e.g. threshold values}
- G01N 29/4436: {with a reference signal (amplitude comparison G01N29/48)}
- G01N 29/46: by spectral analysis, e.g. Fourier analysis {or wavelet analysis (spectral signal processing per se G06F17/14)}
- G01N 2291/269: Various geometry objects
- G01H: MEASUREMENT OF MECHANICAL VIBRATIONS OR ULTRASONIC, SONIC OR INFRASONIC WAVES
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G07C: TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
- G01N: INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES (measuring or testing apparatus or processes other than immunoassay, involving enzymes or microorganisms C12M, C12Q)

## Abstract

Methods, systems, and devices for abnormal sound detection are described. An audio recording of a mechanized environment may be obtained. First sounds extracted from the audio recording may be categorized into a set of categorical sounds. A library of first sound patterns may be generated using the categorical sound and based on second sounds extracted from the audio recording. The first sound patterns may include sequences of the categorical sounds. Audio data including audio signals capture by sensors in the mechanized environment may be received, and second sound patterns detected in the audio signals may be compared with the first sound patterns. Based on comparing the second sound patterns with the first sounds patterns, a sound pattern that is not in the library of the first sound patterns may be identified. An alarm may be generated based on detecting the sound pattern a threshold quantity of times.

## Claims

### Claim 1

A method, comprising:
obtaining an audio recording of a mechanized environment;
categorizing first sounds extracted from the audio recording into a set of categorical sounds;
generating, using the set of categorical sounds, a library of first sound patterns based on second sounds that are extracted from the audio recording, the first sound patterns comprising sequences of categorical sounds of the set of categorical sounds;
receiving audio data comprising audio signals captured by one or more sensors in the mechanized environment;
comparing, based on receiving the audio data, second sound patterns detected in the audio signals with the first sound patterns; and
identifying, based on comparing the second sound patterns with the first sound patterns, a sound pattern of the second sound patterns that is excluded from the library of the first sound patterns.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
determining, based on detecting that the sound pattern of the second sound patterns is excluded from the library of the first sound patterns, that the sound pattern of the second sound patterns has occurred a threshold quantity of times; and
sending, based on detecting that the sound pattern of the second sound patterns has occurred the threshold quantity of times, an alert to inspect the mechanized environment.

### Claim 3 (depends on 2)

The method of claim 2, wherein the threshold quantity of times is based on a length of the sound pattern of the second sound patterns.

### Claim 4 (depends on 1)

The method of claim 1, further comprising:
extracting sounds from the audio recording based on a noise threshold for the mechanized environment, wherein the audio recording comprises a plurality of audio files comprising the sounds, and wherein the sounds comprise the first sounds; and
determining, from the sounds extracted from the audio recording, one or more spectral components for the first sounds.

### Claim 5 (depends on 4)

The method of claim 4, further comprising:
generating a plurality of signal amplitude percentile curves for the plurality of audio files:
identifying, in each of the plurality of signal amplitude percentile curves, a respective elbow feature associated with a respective noise threshold; and
generating a second curve that plots respective noise thresholds determined for each audio file in an ascending order.

### Claim 6 (depends on 5)

The method of claim 5, further comprising:
identifying, in the second curve, an elbow feature associated with the noise threshold; and
determining, based on identifying the elbow feature, the noise threshold, wherein the noise threshold is used for subsequent processing of the audio recording.

### Claim 7 (depends on 4)

The method of claim 4, wherein determining the one or more spectral components for the first sounds comprises:
constructing a multi-segment spectrogram for each sound of the first sounds extracted from the audio recording.

### Claim 8 (depends on 7)

The method of claim 7, wherein constructing the multi-segment spectrogram comprises for a sound of the first sounds comprises:
computing, for the sound of the first sounds, a mean spectrum within each segment of the multi-segment spectrogram.

### Claim 9 (depends on 7)

The method of claim 7, wherein constructing the multi-segment spectrogram for a sound of the first sounds comprises:
determining a first quantity of segments for the sound based on a length of the sound, wherein the multi-segment spectrogram comprises the first quantity of segments and a second quantity of segments:
padding the first quantity of segments, the second quantity of segments, or both, with zeros; and
computing, for the sound, a mean spectrum within each segment of the multi-segment spectrogram.

### Claim 10 (depends on 4)

The method of claim 4, further comprising:
clustering the first sounds into a plurality of clusters based on the one or more spectral components of the first sounds.

### Claim 11 (depends on 10)

The method of claim 10, wherein clustering the sounds further comprises:
applying a one-class support vector machine algorithm to each cluster of the plurality of clusters to filter out a subset of the first sounds identified as noise, wherein a plurality of filtered clusters comprising filtered sounds is obtained based on applying the one-class support vector machine algorithm:
classifying, using a convolutional neural network, the first sounds to obtain a plurality of classifications of the first sounds; and
discarding a second subset of the first sounds that are mismatched between the plurality of clusters and the plurality of classifications.

### Claim 12 (depends on 11)

The method of claim 11, wherein clustering the sounds further comprises:
calculating a plurality of silhouette scores based on the plurality of clusters including different quantities of clusters; and
selecting a quantity for the plurality of clusters associated with a highest silhouette score of the plurality of silhouette scores.

### Claim 13 (depends on 10)

The method of claim 10, wherein the first sounds are clustered into the plurality of clusters using an unsupervised clustering procedure.

### Claim 14 (depends on 10)

The method of claim 10, wherein categorizing the first sounds into the set of categorical sounds comprises;
assigning identifiers of a plurality of identifiers to filtered clusters of the plurality of clusters, wherein the identifiers correspond to categorical sounds of the set of categorical sounds, and wherein respective subsets of the first sounds included in respective filtered clusters are labeled with respective identifiers assigned to the respective filtered clusters to obtain labeled sounds.

### Claim 15 (depends on 14)

The method of claim 14, further comprising:
training a convolutional neural network using the labeled sounds to obtain a trained convolutional neural network;
extracting, based on training the convolutional neural network, the second sounds from the audio recording; and
inputting the second sounds into the trained convolutional neural network, wherein the second sounds are categorized into the set of categorical sounds based on the second sounds being inputted into the trained convolutional neural network.

### Claim 16 (depends on 1)

The method of claim 1, wherein generating the library of the first sound patterns comprises:
identifying each sound in respective subsets of the second sounds, the second sounds comprising one or more idle periods that correspond to one or more quiet periods between categorical sounds, wherein the set of categorical sounds comprises an idle sound associated with idle periods between categorical sounds;
determining, for each sound in the respective subsets of the second sounds, a respective magnitude and a respective categorical sound:
identifying, in the second sounds, the sequences of categorical sounds of the set of categorical sounds that repeat a threshold quantity of times; and
storing, in the library of the first sound patterns, the sequences of categorical sounds that occur the threshold quantity of times.

### Claim 17 (depends on 16)

The method of claim 16, further comprising:
detecting the second sound patterns, wherein detecting the second sound patterns comprises:
identifying, in third sounds, each sound in respective subsets of the third sounds that are extracted from the audio signals captured from the mechanized environment,
determining, for each sound in the respective subsets of the third sounds, the respective magnitude and the respective categorical sound, and
identifying, in the third sounds, second sequences of categorical sounds of the set of categorical sounds.

### Claim 18 (depends on 1)

The method of claim 1, wherein generating the library of the first sound patterns comprises:
applying an n-gram natural language processing method to the first sounds to obtain a plurality of n-grams.

### Claim 19

An apparatus, comprising:
one or more processors; and
memory storing instructions that are executable by the one or more processors to cause the apparatus to:
obtain an audio recording of a mechanized environment:
categorize first sounds extracted from the audio recording into a set of categorical sounds;
generate, using the set of categorical sounds, a library of first sound patterns based on second sounds that are extracted from the audio recording, the first sound patterns comprising sequences of categorical sounds of the set of categorical sounds;
receive audio data comprising audio signals captured by one or more sensors in the mechanized environment:
compare, based on receiving the audio data, second sound patterns detected in the audio signals with the first sound patterns; and
identify, based on comparing the second sound patterns with the first sound patterns, a sound pattern of the second sound patterns that is excluded from the library of the first sound patterns.

### Claim 20

A non-transitory, computer-readable medium that stores code that comprises instructions executable by a processor of an electronic device to cause the electronic device to:
obtain an audio recording of a mechanized environment:
categorize first sounds extracted from the audio recording into a set of categorical sounds;
generate, using the set of categorical sounds, a library of first sound patterns based on second sounds that are extracted from the audio recording, the first sound patterns comprising sequences of categorical sounds of the set of categorical sounds:
receive audio data comprising audio signals captured by one or more sensors in the mechanized environment;
compare, based on receiving the audio data, second sound patterns detected in the audio signals with the first sound patterns; and
identify, based on comparing the second sound patterns with the first sound patterns, a sound pattern of the second sound patterns that is excluded from the library of the first sound patterns.