# US 2021/0277564 A1 · System and method for using sound to monitor the operation of a washing machine appliance > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0277564 A1 - Document ID: US20210277564A1 - Status: Published application - Kind: A1 - Application number: 16/807,269 - Filed: 2020-03-03 - Published: 2021-09-09 - Earliest claimed: 2020-03-03 - Assignees: Haier US Appliance Solutions, Inc. - Applicants: Haier US Appliance Solutions, Inc. - Inventors: Khalid Jamal Mashal; David Scott Dunn; Juan Manuel Huerta - Web page: https://patentgrep.com/patent/US20210277564A1 ## CPC - D06F 34/14: Arrangements for detecting or measuring specific parameters - D06F 33/32: Control of operational steps, e.g. optimisation or improvement of operational steps depending on the condition of the laundry - D06F 33/34: of water filling - D06F 33/40: of centrifugal separation of water from the laundry - D06F 33/44: Control of the operating time, e.g. reduction of overall operating time - D06F 34/04: Signal transfer or data transmission arrangements - D06F 2103/00: Parameters monitored or detected for the control of domestic laundry washing machines, washer-dryers or laundry dryers - D06F 2103/18: Washing liquid level - D06F 2103/24: Spin speed; Drum movements - D06F 2103/26: Imbalance; Noise level - D06F 2105/52: Changing sequence of operational steps; Carrying out additional operational steps; Modifying operational steps, e.g. by extending duration of steps - D06F: LAUNDERING, DRYING, IRONING, PRESSING OR FOLDING TEXTILE ARTICLES ## Abstract A washing machine appliance includes a microphone for monitoring sound generated during operation of the washing machine appliance and a controller is operably coupled to the microphone. The controller is configured for obtaining a sound signal generated during operation of the washing machine appliance and converting the sound signal into a spectrogram that represents a sound frequency and a sound amplitude over time. An artificial intelligence image recognition process is used to analyze the spectrogram to identify one or more sound signatures that are associated with particular operating conditions, and operation of the washing machine appliance is adjusted based at least in part on the identification of the sound signature. ## Claims ### Claim 1 A washing machine appliance comprising: a wash tub positioned within a cabinet and defining a wash chamber; a wash basket rotatably mounted within the wash tub and being configured for receiving of a load of articles for washing; a motor operably coupled to the wash basket for selectively rotating the wash basket; a microphone for monitoring sound generated during operation of the washing machine appliance; and a controller operably coupled to the microphone, the controller being configured for: obtaining a sound signal generated during operation of the washing machine appliance using the microphone; generating a spectrogram from the sound signal, the spectrogram representing a sound frequency and a sound amplitude over time; identifying a sound signature by analyzing the spectrogram using an image recognition process; and adjusting at least one operating parameter of the washing machine appliance based at least in part on the identification of the sound signature. ### Claim 2 (depends on 1) The washing machine appliance of claim 1, wherein the image recognition process uses artificial intelligence (AI) to analyze the spectrogram. ### Claim 3 (depends on 1) The washing machine appliance of claim 1, wherein the image recognition process comprises a convolution neural network (CNN). ### Claim 4 (depends on 1) The washing machine appliance of claim 1, wherein the controller is further configured for: learning a plurality of sound signatures associated with various operating conditions. ### Claim 5 (depends on 1) The washing machine appliance of claim 1, wherein the sound signature is associated with sounds generated from at least one of a bearing, a belt, the motor, a water valve, a pump, a suspension system, harmonics of structural components, or undesirable contact between components or subsystems. ### Claim 6 (depends on 1) The washing machine appliance of claim 1, wherein adjusting the at least one operating parameter comprises: adjusting an agitation time or profile, adjusting a water level, limiting a spin speed, identifying service needs, or providing a user with operating guidance. ### Claim 7 (depends on 1) The washing machine appliance of claim 1, wherein adjusting the at least one operating parameter comprises: selecting an operating cycle based on the sound signature. ### Claim 8 (depends on 1) The washing machine appliance of claim 1, wherein the controller is further configured for: providing a user notification when the sound signature indicates that a predetermined operating characteristic exists. ### Claim 9 (depends on 1) The washing machine appliance of claim 1, wherein the sound signature is associated with the presence of an unwashable item, and wherein adjusting the at least one operating parameter comprises stopping the wash cycle. ### Claim 10 (depends on 1) The washing machine appliance of claim 1, wherein the controller is further configured for: transmitting the spectrogram to a remote server for analysis; and receiving analytic feedback from the remote server. ### Claim 11 (depends on 1) The washing machine appliance of claim 1, wherein the microphone is positioned outside the cabinet and remote from the washing machine appliance. ### Claim 12 A method of operating a washing machine appliance, the washing machine appliance comprising a wash basket rotatably mounted within a wash tub, a motor operably coupled to the wash basket for selectively rotating the wash basket, and a microphone for monitoring sound generated by the washing machine appliance, the method comprising: obtaining a sound signal generated during operation of the washing machine appliance using the microphone; generating a spectrogram from the sound signal, the spectrogram representing a sound frequency and a sound amplitude over time; identifying a sound signature by analyzing the spectrogram using an image recognition process; and adjusting at least one operating parameter of the washing machine appliance based at least in part on the identification of the sound signature. ### Claim 13 (depends on 12) The method of claim 12, wherein the image recognition process uses artificial intelligence (AI) to analyze the spectrogram. ### Claim 14 (depends on 12) The method of claim 12, wherein the image recognition process comprises a convolution neural network (CNN). ### Claim 15 (depends on 12) The method of claim 12, further comprising: learning a plurality of sound signatures associated with various operating conditions. ### Claim 16 (depends on 12) The method of claim 12, wherein the sound signature is associated with the presence of an unwashable item, and wherein adjusting the at least one operating parameter comprises stopping the wash cycle. ### Claim 17 (depends on 12) The method of claim 12, further comprising: transmitting the spectrogram to a remote server for analysis; and receiving analytic feedback from the remote server. ### Claim 18 An appliance comprising: a microphone for monitoring sound generated during operation of the appliance; and a controller operably coupled to the microphone, the controller being configured for: obtaining a sound signal generated during operation of the appliance using the microphone; generating a spectrogram from the sound signal, the spectrogram representing a sound frequency and a sound amplitude over time; identifying a sound signature by analyzing the spectrogram using an image processing technique; and adjusting at least one operating parameter of the appliance based at least in part on the identification of the sound signature. ### Claim 19 (depends on 18) The appliance of claim 18, wherein the image recognition process uses artificial intelligence (AI) to analyze the spectrogram. ### Claim 20 (depends on 18) The appliance of claim 18, wherein the image recognition process comprises a convolution neural network (CNN).