Back to results
US 12,746,625 B2Granted
Multi-resolution audio defect detection in welding
AssigneeInternational Business Machines Corporation
InventorsLaura Wynter, Penny Chong
Life of this patent
Estimating the term…
Abstract
According to at least one embodiment, a method, a computer system, and a computer program product for multi-resolution audio defect detection in welding is provided. The present invention may include receiving unlabeled and labeled audio data; formatting the received unlabeled and labeled audio data; training a foundation model using the formatted unlabeled audio data in a self-supervised manner with a reconstruction loss; training the foundation model using the formatted labeled audio data with a classification loss; and performing multi-resolution audio defect detection on welding audio data using the trained foundation model.
Claims20
A processor-implemented method for multi-resolution audio defect detection in welding, the method comprising:
- receiving unlabeled and labeled audio data recorded from welding processes;
- formatting the received unlabeled and labeled audio data, wherein the formatting comprises:
- transforming the labeled and unlabeled audio data into a Mel spectrogram;
- splitting the Mel spectrogram into 2D patches;
- learning importance scores for the 2D patches based on the unlabeled audio data;
- generating a coarse-grained masked Mel spectrogram, wherein the generating the coarse-grained masked Mel spectrogram comprises masking at least one of the 2D patches based on the learned importance scores;
- and
- masking a set of fine-grained Mel spectrograms selected from the coarse-grained masked Mel spectrogram based on the learned importance scores to obtain fine-grained masked Mel spectrograms;
- pretraining a foundation model using the formatted unlabeled audio data in a self-supervised manner with a reconstruction loss;
- fine-tuning the pretrained foundation model using the formatted labeled audio data with a classification loss to obtain a trained foundation model;
- formatting new welding audio data recorded from a welding process;
- and
- performing multi-resolution audio defect detection on the formatted new welding audio data using the trained foundation model.
10 more elements
A computer system for multi-resolution audio defect detection in welding, the computer system comprising:
- one or more processors, one or more computer-readable memories, one or more computer-readable tangible storage medium, and program instructions stored on at least one of the one or more tangible storage medium for execution by at least one of the one or more processors via at least one of the one or more memories, wherein the computer system is capable of performing a method comprising:
- receiving unlabeled and labeled audio data recorded from welding processes;
- formatting the received unlabeled and labeled audio data, wherein the formatting comprises:
- transforming the labeled and unlabeled audio data into a Mel spectrogram;
- splitting the Mel spectrogram into 2D patches;
- learning importance scores for the 2D patches based on the unlabeled audio data;
- generating a coarse-grained masked Mel spectrogram, wherein the generating the coarse-grained masked Mel spectrogram comprises masking at least one of the 2D patches based on the learned importance scores;
- and
- masking a set of fine-grained Mel spectrograms selected from the coarse-grained masked Mel spectrogram based on the learned importance scores to obtain fine-grained masked Mel spectrograms;
- pretraining a foundation model using the formatted unlabeled audio data in a self-supervised manner with a reconstruction loss;
- fine-tuning the pretrained foundation model using the formatted labeled audio data with a classification loss to obtain a trained foundation model;
- formatting new welding audio data recorded from a welding process;
- and
- performing multi-resolution audio defect detection on the formatted new welding audio data using the trained foundation model.
11 more elements
A computer program product for multi-resolution audio defect detection in welding, the computer program product comprising:
- one or more computer-readable tangible storage medium and program instructions stored on at least one of the one or more tangible storage medium, the program instructions executable by a processor to cause the processor to perform a method comprising:
- receiving unlabeled and labeled audio data recorded from welding processes;
- formatting the received unlabeled and labeled audio data, wherein the formatting comprises:
- transforming the labeled and unlabeled audio data into a Mel spectrogram;
- splitting the Mel spectrogram into 2D patches:
- learning importance scores for the 2D patches based on the unlabeled audio data;
- generating a coarse-grained masked Mel spectrogram, wherein the generating the coarse-grained masked Mel spectrogram comprises masking at least one of the 2D patches based on the learned importance scores;
- and
- masking a set of fine-grained Mel spectrograms selected from the coarse-grained masked Mel spectrogram based on the learned importance scores to obtain fine-grained masked Mel spectrograms;
- pretraining a foundation model using the formatted unlabeled audio data in a self-supervised manner with a reconstruction loss;
- fine-tuning the pretrained foundation model using the formatted labeled audio data with a classification loss to obtain a trained foundation model;
- formatting new welding audio data recorded from a welding process;
- and
- performing multi-resolution audio defect detection on the formatted new welding audio data using the trained foundation model.
11 more elements
Dependencies are inferred from the claim text.