# US 12,014,749 B2 · Audio samples to detect device anomalies

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,014,749 B2
- Document ID: US12014749B2
- Status: Granted
- Kind: B2
- Application number: 17/783,305
- Filed: 2020-01-10
- Published: 2023-01-12
- Granted: 2024-06-18
- Earliest claimed: 2020-01-10
- Assignees: Hewlett-Packard Development Company, L.P.
- Applicants: Hewlett-Packard Development Company, L.P.
- Inventors: Anton Wiranata; Kathryn Janet Ferguson; Mark Q. Shaw; Chin-Ning Chen; Jan Allebach
- Web page: https://patentgrep.com/patent/US12014749B2

## CPC

- G10L 25/27: characterised by the analysis technique
- G10L 25/51: for comparison or discrimination
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

Example implementations relate to audio samples to detect device anomalies. For example, computing device, comprising: a processing resource and a non-transitory computer readable medium storing instructions executable by the processing resource to: generate a matrix of audio information for a plurality of audio samples of a device, select audio information from one of the plurality of audio samples, generate a plurality of principal components for the selected audio information utilizing a principal component expansion, select a principal component from the plurality of principal components based on a quantity of variance, and detect an anomaly of the device based on a comparison between a real time audio sample of the device and the selected principal component.

## Claims

### Claim 1

A computing device, comprising:
a processing resource;
a non-transitory computer readable medium storing instructions executable by the processing resource to:
generate a matrix of audio information for a plurality of audio samples of a device;
select audio information from one of the plurality of audio samples;
generate a plurality of principal components for the selected audio information utilizing a principal component analysis (PCA);
select a principal component from the plurality of principal components based on a quantity of variance; and
detect an anomaly of the device based on a comparison between a real time audio sample of the device and the selected principal component.

### Claim 2 (depends on 1)

The computing device of claim 1, wherein the matrix of audio information includes original audio samples of the device and augmented audio samples of the device.

### Claim 3 (depends on 2)

The computing device of claim 2, wherein the matrix of audio information includes a first portion that includes audio information for the original audio samples and a second portion augmented under the first portion that includes audio information for the augmented audio samples.

### Claim 4 (depends on 1)

The computing device of claim 1, wherein the selected principal component includes a greater quantity of variance than the remaining plurality of principal components.

### Claim 5 (depends on 4)

The computing device of claim 4, wherein the selected principal component represents the audio information for the plurality of audio samples.

### Claim 6 (depends on 1)

The computing device of claim 1, comprising instructions executable by the processing resource to input the selected principal component within an anomalous detection device.

### Claim 7 (depends on 1)

The computing device of claim 1, comprising instructions executable by the processing resource to generate a plurality of principal components for each of the remaining plurality of audio samples utilizing the principal component analysis.

### Claim 8

A non-transitory computer-readable storage medium comprising instructions when executed cause a processor of a computing device to:
generate a matrix of audio information for a plurality of audio samples of a device collected at a time when the device is operating within a set of specifications, wherein the matrix includes a first portion that includes original audio samples and a second portion appended under the first portion that includes augmented audio samples;
generate a plurality of principal components for each of the plurality of audio samples of the matrix utilizing a principal component analysis;
select a principal component from the plurality of principal components based on a quantity of variance within each of the plurality of principal components; and
input the selected principal component into a detection model to determine when a real time audio sample of the device exceeds a threshold defined by the detection model.

### Claim 9 (depends on 8)

The medium of claim 8, wherein the audio information includes a Discrete Tone Frequency, a Power at Discrete Tone Frequency Relative to Average, a Power at Discrete Tone Frequency, a power spectral density (PSD) peak width, a modulation frequency, and a modulation depth percentage.

### Claim 10 (depends on 8)

The medium of claim 8, wherein the second portion includes a pitch shift of an original audio file, a time stretch of the original audio file, and a mixture of augmentation of the original file.

### Claim 11 (depends on 8)

The medium of claim 8, wherein the matrix includes a plurality of columns that represent a feature vector of the plurality of audio samples and rows of the plurality of audio samples.

### Claim 12 (depends on 8)

The medium of claim 8, comprising instructions when executed cause the processor of the computing device to determine a mean vector of the matrix and a covariance matrix of the matrix.

### Claim 13

A system comprising:
a sound recording device to capture a plurality of original audio samples of a printing device while the printing device is operating within manufacturer specifications;
a computing device comprising a processor and a memory storing instructions executable by the processor to:
augment the original audio samples to generate a plurality of augmented audio samples simulating to increase a quantity of variance among a plurality of audio samples including the original audio samples and the augmented audio samples;
generate a first matrix of the original audio samples of the printing device by extracting features from each original audio sample, wherein the first matrix has a plurality of rows corresponding to the original audio samples and a plurality of columns corresponding to the extracted features;
generate a second matrix of the augmented audio samples of the printing device by extracting the features from each augmented audio sample, wherein the second matrix has a plurality of rows corresponding to the augmented audio samples and a plurality of columns corresponding to the extracted features, the columns of the second matrix respectively corresponding to the columns of the first matrix;
append the second matrix to a bottom of the first matrix to generate a third matrix having a plurality of rows including the rows of the first matrix and the rows of the second matrix and having a plurality of columns respectively corresponding to the columns of the first matrix;
generate a plurality of principal components for the third matrix utilizing a principal component analysis, by performing the principal component analysis for each row of the third matrix; and
select the principal component having a greatest quantity of variance; and
an anomalous detection device to:
receive the selected principal component; and
utilize the selected principal component to determine when a real time audio sample exceeds a threshold.

### Claim 14 (depends on 13)

The system of claim 13, wherein the threshold is based on a variance of the selected principal component.

### Claim 15 (depends on 13)

The system of claim 13, wherein the anomalous detection device utilizes a one class support vector machine (OCSVM) or random forests (RF) model to determine when the real time audio sample exceeds the threshold.