# US 12,602,195 B2 · Method and apparatus for controlling device based on tapping and storage medium > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,602,195 B2 - Document ID: US12602195B2 - Status: Granted - Kind: B2 - Application number: 18/830,252 - Filed: 2024-09-10 - Published: 2024-12-26 - Granted: 2026-04-14 - Earliest claimed: 2022-03-11 - Assignees: Haier Smart Home Co., Ltd.; Qingdao Haier Washing Machine Co., Ltd. - Applicants: Qingdao Haier Washing Machine Co., Ltd.; Haier Smart Home Co., Ltd. - Inventors: Xiaopeng Ding; Zhenxing Huang; Sheng Xu - Web page: https://patentgrep.com/patent/US12602195B2 ## CPC - G06F 3/16: Sound input; Sound output (speech processing G10L) - G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback} - G06F 18/24: Classification techniques - G06N 3/09: Supervised learning - G10L 25/51: for comparison or discrimination - G06F 2200/1636: Sensing arrangement for detection of a tap gesture on the housing - G10L 25/30: using neural networks - Y02P 90/02: Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS] - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - Y02P: CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS ## Abstract The present application provides a method and apparatus for controlling a device based on tapping and a storage medium. The method includes: in response to a tapping action from a user, acquiring sound information triggered by the tapping action; performing recognition processing on the sound information according to a preset recognition model set to determine a recognition result of the sound information, where the recognition model set includes at least one recognition model; and controlling the device according to the recognition result. No components needs to be added for the method, application limitations of the method no longer exist, and the diverse control of the device can be realized. ## Claims ### Claim 1 A method for controlling a device based on tapping, comprising: in response to a tapping action from a user, acquiring sound information triggered by the tapping action; performing recognition processing on the sound information according to a preset recognition model set to determine a recognition result of the sound information, wherein the recognition model set comprises at least one recognition model; controlling the device according to the recognition result, wherein the performing recognition processing on the sound information according to a preset recognition model set to determine a recognition result of the sound information comprises: determining tapping dimension information of the sound information; determining a corresponding recognition model according to the tapping dimension information of the sound information, and performing recognition processing on the sound information according to each recognition model to determine a classification recognition result of the sound information by each recognition model, wherein the sound information corresponds to at least one recognition model, and each recognition model corresponds to at least one tapping dimension information; and performing merging processing on each classification recognition result to determine the recognition result of the sound information. ### Claim 2 (depends on 1) The method according to claim 1, wherein the performing recognition processing on the sound information according to each recognition model to determine a classification recognition result of the sound information by each recognition model comprises: performing recognition processing on the sound information according to each recognition model to determine a confidence rate of the sound information under each recognition model, wherein the confidence rate represents a probability of the sound information being recognized by a recognition model; if the confidence rate is greater than or equal to a preset threshold, determining a classification recognition result of the sound information from the corresponding recognition model to be that the sound information passes recognition; if the confidence rate is less than a preset threshold, determining a classification recognition result of the sound information from the corresponding recognition model to be that the sound information does not pass recognition. ### Claim 3 (depends on 1) The method according to claim 1, wherein the tapping dimension information comprises one or more of tapping frequency information, tapping force information, tapping position information, information about a driving distance of tapping and information about a contact area of tapping. ### Claim 4 (depends on 1) The method according to claim 1, wherein the controlling the device according to the recognition result comprises: if the recognition result indicates that the sound information does not pass a recognition by any recognition model, controlling the device to output prompting information, wherein the prompting information is used for prompting the user to reenter a tapping action; if the recognition result indicates that the sound information passes a recognition by one or more recognition models, outputting a control instruction according to the recognition result, and controlling the device to execute the corresponding control instruction, wherein the control instruction is used for controlling the device to perform one or more of starting up, shutting down, intelligent interaction, door opening and door closing. ### Claim 5 (depends on 1) The method according to claim 1, wherein the method further comprises: acquiring multi-dimensional training audio information, wherein the multi-dimensional training audio information comprises audio information at different tapping positions, audio information over different tapping frequencies, audio information under different tapping forces, audio information for different driving distances of tapping and audio information for different contact areas of tapping; conducting classification model training for the multi-dimensional training audio information according to a preset neural network model to determine the recognition model set, wherein different recognition models have different recognition dimensions. ### Claim 6 (depends on 1) The method according to claim 1, wherein the method further comprises: acquiring multi-dimensional training audio information, wherein the multi-dimensional training audio information comprises audio information at different tapping positions, audio information over different tapping frequencies, audio information under different tapping forces, audio information for different driving distances of tapping and audio information for different contact areas of tapping; conducting classification model training for the multi-dimensional training audio information according to a preset neural network model to determine the recognition model set, wherein different recognition models have different recognition dimensions. ### Claim 7 (depends on 2) The method according to claim 2, wherein the method further comprises: acquiring multi-dimensional training audio information, wherein the multi-dimensional training audio information comprises audio information at different tapping positions, audio information over different tapping frequencies, audio information under different tapping forces, audio information for different driving distances of tapping and audio information for different contact areas of tapping; conducting classification model training for the multi-dimensional training audio information according to a preset neural network model to determine the recognition model set, wherein different recognition models have different recognition dimensions. ### Claim 8 (depends on 3) The method according to claim 3, wherein the method further comprises: acquiring multi-dimensional training audio information, wherein the multi-dimensional training audio information comprises audio information at different tapping positions, audio information over different tapping frequencies, audio information under different tapping forces, audio information for different driving distances of tapping and audio information for different contact areas of tapping; conducting classification model training for the multi-dimensional training audio information according to a preset neural network model to determine the recognition model set, wherein different recognition models have different recognition dimensions. ### Claim 9 (depends on 4) The method according to claim 4, wherein the method further comprises: acquiring multi-dimensional training audio information, wherein the multi-dimensional training audio information comprises audio information at different tapping positions, audio information over different tapping frequencies, audio information under different tapping forces, audio information for different driving distances of tapping and audio information for different contact areas of tapping; conducting classification model training for the multi-dimensional training audio information according to a preset neural network model to determine the recognition model set, wherein different recognition models have different recognition dimensions. ### Claim 10 An electronic device, comprising: at least one processor; and a memory communicatively connected with the at least one processor; wherein the memory stores an instruction executable by the at least one processor, and the instruction is executed by the at least one processor to enable the at least one processor to: acquire sound information triggered by a tapping action in response to the tapping action from a user; perform recognition processing on the sound information according to a preset recognition model set to determine a recognition result of the sound information, wherein the recognition model set comprises at least one recognition model; control the device according to the recognition result, wherein the at least one processor is specifically enabled to: determine tapping dimension information of the sound information; determine a corresponding recognition model according to the tapping dimension information of the sound information, and perform recognition processing on the sound information according to each recognition model to determine a classification recognition result of the sound information by each recognition model, wherein the sound information corresponds to at least one recognition model, and each recognition model corresponds to at least one tapping dimension information; and perform merging processing on each classification recognition result to determine the recognition result of the sound information. ### Claim 11 (depends on 10) The electronic device according to claim 10, wherein the at least one processor is enabled to: perform recognition processing on the sound information according to each recognition model to determine a confidence rate of the sound information under each recognition model, wherein the confidence rate represents a probability of the sound information being recognized by a recognition model; if the confidence rate is greater than or equal to a preset threshold, determine a classification recognition result of the sound information from the corresponding recognition model to be that the sound information passes recognition; if the confidence rate is less than a preset threshold, determine a classification recognition result of the sound information from the corresponding recognition model to be that the sound information does not pass recognition. ### Claim 12 (depends on 10) The electronic device according to claim 10, wherein the tapping dimension information comprises one or more of tapping frequency information, tapping force information, tapping position information, information about a driving distance of tapping and information about a contact area of tapping. ### Claim 13 (depends on 10) The electronic device according to claim 10, wherein the at least one processor is enabled to: if the recognition result indicates that the sound information does not pass a recognition by any recognition model, control the device to output prompting information, wherein the prompting information is used for prompting the user to reenter a tapping action; if the recognition result indicates that the sound information passes a recognition by one or more recognition models, output a control instruction according to the recognition result, and control the device to execute the corresponding control instruction, wherein the control instruction is used for controlling the device to perform one or more of starting up, shutting down, intelligent interaction, door opening and door closing. ### Claim 14 (depends on 10) The electronic device according to claim 10, wherein the at least one processor is enabled to: acquire multi-dimensional training audio information, wherein the multi-dimensional training audio information comprises audio information at different tapping positions, audio information over different tapping frequencies, audio information under different tapping forces, audio information for different driving distances of tapping and audio information for different contact areas of tapping; conduct classification model training for the multi-dimensional training audio information according to a preset neural network model to determine the recognition model set, wherein different recognition models have different recognition dimensions. ### Claim 15 (depends on 10) The electronic device according to claim 10, wherein the at least one processor is enabled to: acquire multi-dimensional training audio information, wherein the multi-dimensional training audio information comprises audio information at different tapping positions, audio information over different tapping frequencies, audio information under different tapping forces, audio information for different driving distances of tapping and audio information for different contact areas of tapping; conduct classification model training for the multi-dimensional training audio information according to a preset neural network model to determine the recognition model set, wherein different recognition models have different recognition dimensions. ### Claim 16 (depends on 11) The electronic device according to claim 11, wherein the at least one processor is enabled to: acquire multi-dimensional training audio information, wherein the multi-dimensional training audio information comprises audio information at different tapping positions, audio information over different tapping frequencies, audio information under different tapping forces, audio information for different driving distances of tapping and audio information for different contact areas of tapping; conduct classification model training for the multi-dimensional training audio information according to a preset neural network model to determine the recognition model set, wherein different recognition models have different recognition dimensions. ### Claim 17 (depends on 12) The electronic device according to claim 12, wherein the at least one processor is enabled to: acquire multi-dimensional training audio information, wherein the multi-dimensional training audio information comprises audio information at different tapping positions, audio information over different tapping frequencies, audio information under different tapping forces, audio information for different driving distances of tapping and audio information for different contact areas of tapping; conduct classification model training for the multi-dimensional training audio information according to a preset neural network model to determine the recognition model set, wherein different recognition models have different recognition dimensions. ### Claim 18 A non-transitory computer readable storage medium, wherein the computer readable storage medium is stored with a computer executable instruction, wherein when the computer executable instruction is executed by a processor to: acquire sound information triggered by a tapping action in response to the tapping action from a user; perform recognition processing on the sound information according to a preset recognition model set to determine a recognition result of the sound information, wherein the recognition model set comprises at least one recognition model; control the device according to the recognition result, wherein the processor is specifically enabled to: determine tapping dimension information of the sound information; determine a corresponding recognition model according to the tapping dimension information of the sound information, and perform recognition processing on the sound information according to each recognition model to determine a classification recognition result of the sound information by each recognition model, wherein the sound information corresponds to at least one recognition model, and each recognition model corresponds to at least one tapping dimension information; and perform merging processing on each classification recognition result to determine the recognition result of the sound information.