# US 2023/0274757 A1 · Apparatus and method for inspection > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2023/0274757 A1 - Document ID: US20230274757A1 - Status: Published application - Kind: A1 - Application number: 17/901,597 - Filed: 2022-09-01 - Published: 2023-08-31 - Earliest claimed: 2022-02-28 - Assignees: Hitachi, Ltd. - Applicants: Hitachi, Ltd. - Inventors: Atsushi OKU; Kazuya SUMITA; Kozue ONISHI; Akihiro MATSUMOTO; Eri KUBO; Keitaro UEHARA - Web page: https://patentgrep.com/patent/US20230274757A1 ## CPC - G10L 25/51: for comparison or discrimination - G10L 25/18: the extracted parameters being spectral information of each sub-band - G10L 25/30: using neural networks - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING ## Abstract An apparatus and/or method is proposed that can show evidence for judgment results in an abnormality judgment using a judgment model obtained by machine learning. Pieces of processed data with a mask corresponding to characteristics of waveform data set on a spectrogram of waveform data are created using a judgment model obtained by machine learning by sequentially shifting the mask in a direction corresponding to the mask, a change rate or a change degree of the waveform data of each piece of created processed data from the spectrogram is calculated, each area in which the mask on the spectrogram of the waveform data is set based on the calculated change rate or change degree is colored with a color or concentration corresponding to the change rate or change degree of the processed data when the mask is set so as to draw and display a judgment evidence image. ## Claims ### Claim 1 An inspection apparatus that judges the presence or absence of abnormality based on a spectrogram of waveform data, comprising: an inspection judgment section that calculates anomaly score of the spectrogram of the waveform data using a judgment model obtained by machine learning and judges the presence or absence of abnormality based on the calculated anomaly score; a processed data creation section that creates a plurality of pieces of processed data with a mask corresponding to characteristics of the waveform data set on the spectrogram of the waveform data so as to sequentially shift the mask in a direction corresponding to the mask; an anomaly score/change degree calculation section that calculates anomaly score of the processed data created by the processed data creation section and calculates each change rate or change degree of the waveform data of the processed data from the spectrogram based on the anomaly score of the calculated processed data and the anomaly score of the spectrogram of the waveform data calculated by the inspection judgment section; a judgment evidence drawing section that draws a judgment evidence image obtained by coloring each area in which the mask on the spectrogram of the waveform data is set based on the change rate or change degree of the calculated processed data from the spectrogram of the waveform data, with a color or concentration corresponding to the change rate or change degree of the processed data when the mask is set; and a result display section that displays the judgment result of the inspection judgment section and the judgment evidence image drawn by the judgment evidence drawing section. ### Claim 2 (depends on 1) The inspection apparatus according to claim 1, wherein the processed data creation section creates the processed data using at least one of an elongated rectangular first mask over an entire range in a frequency direction parallel to the frequency direction of the spectrogram and an elongated rectangular second mask over an entire range in a time direction parallel to the time direction of the spectrogram as the mask. ### Claim 3 (depends on 2) The inspection apparatus according to claim 2, further comprising a mask creation section that calculates a width in the time direction of an area on the spectrogram of the waveform data where features of the waveform data appear as a mask width of the first mask and calculates a width in the frequency direction of the area on the spectrogram of the waveform data where the features of the waveform data appear as a mask width of the second mask, wherein the processed data creation section creates the processed data using the mask with the mask width calculated from the mask creation section. ### Claim 4 (depends on 3) The inspection apparatus according to claim 3, wherein the mask creation section determines a mask value of the mask at each position where the mask on the spectrogram is set based on an average value of the data value of each area part in which the mask on the spectrogram of the waveform data should be set, and the processed data creation section creates the processed data using the mask with the mask value determined by the mask creation section. ### Claim 5 (depends on 4) The inspection apparatus according to claim 4, wherein when an average value of data value of each area part in which the mask on the spectrogram of the waveform data provided in advance is set is equal to or larger than a predetermined threshold, the mask creation section determines the average value as the mask value of the mask, and when the average value is smaller than the threshold, the mask creation section determines a predetermined value as the mask value of the mask. ### Claim 6 (depends on 1) The inspection apparatus according to claim 1, wherein when the judgment model is an auto encoder machine learning model, the anomaly score/change degree calculation section calculates an average value of abnormality value of each pixel in unprocessed areas other than the area of the spectrogram of the waveform data in which the mask is set as an abnormality value of the spectrogram of the waveform data when the mask is set in the area. ### Claim 7 (depends on 2) The inspection apparatus according to claim 2, wherein the processed data creation section creates the processed data using the first and second masks with the fixed mask width. ### Claim 8 An inspection method executed by an inspection apparatus that judges the presence or absence of abnormality based on a spectrogram of waveform data, the method comprising: a first step of calculating anomaly score of the spectrogram of the waveform data using a judgment model obtained by machine learning and judging the presence or absence of abnormality based on the calculated anomaly score; a second step of creating a plurality of pieces of processed data with a mask corresponding to characteristics of the waveform data set on the spectrogram of the waveform data so as to sequentially shift the mask in a direction corresponding to the mask; a third step of calculating anomaly score of the created processed data and calculating each change rate or change degree of the waveform data of the processed data from the spectrogram based on the calculated anomaly score of processed data and the calculated anomaly score of the spectrogram of the waveform data; a fourth step of drawing a judgment evidence image obtained by coloring each area in which the mask on the spectrogram of the waveform data is set based on the change rate or change degree of the calculated processed data from the spectrogram of the waveform data, with a color or concentration corresponding to the change rate or change degree of the processed data when the mask is set; and a fifth step of displaying the judgment result about the presence or absence of the abnormality and the drawn judgment evidence image. ### Claim 9 (depends on 8) The inspection method according to claim 8, wherein in the second step, the inspection apparatus creates the processed data using at least one of an elongated rectangular first mask over an entire range in a frequency direction parallel to the frequency direction of the spectrogram and an elongated rectangular second mask over an entire range in a time direction parallel to the time direction of the spectrogram as the mask. ### Claim 10 (depends on 9) The inspection method according to claim 9, wherein in the second step, the inspection apparatus calculates a width in the time direction of an area on the spectrogram of the waveform data where features of the waveform data appear as the mask width of the first mask and calculates a width in the frequency direction of the area on the spectrogram of the waveform data where the features of the waveform data appear as the mask width of the second mask, and the processed data is created using the mask with the calculated mask width. ### Claim 11 (depends on 10) The inspection method according to claim 10, wherein in the second step, the inspection apparatus determines the mask value of the mask at each position where the mask on the spectrogram is set based on an average value of the data value of each area part for which the mask on the spectrogram of the waveform data should be set, and the processed data is created using the mask with the determined mask value. ### Claim 12 (depends on 11) The inspection method according to claim 11, wherein in the second step, when an average value of data value of each area part in which the mask on the spectrogram of the waveform data provided in advance is equal to or larger than a predetermined threshold, the inspection apparatus determines the average value as the mask value of the mask, and when the average value is smaller than the threshold, the inspection apparatus determines a predetermined value as the mask value of the mask. ### Claim 13 (depends on 8) The inspection method according to claim 8, wherein in the third step, when the judgment model is an auto encoder machine learning model, the inspection apparatus calculates an average value of abnormality value at each pixel in unprocessed areas other than the area of the spectrogram of the waveform data in which the mask is set as an abnormality value of the spectrogram of the waveform data when the mask is set in the area. ### Claim 14 (depends on 9) The inspection method according to claim 9, wherein in the second step, the inspection apparatus creates the processed data using the first and second masks with the fixed mask width.