# US 2021/0280179 A1 · Household appliance control method, device and system, and intelligent air conditioner

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0280179 A1
- Document ID: US20210280179A1
- Status: Published application
- Kind: A1
- Application number: 16/330,642
- Filed: 2017-02-14
- Published: 2021-09-09
- Earliest claimed: 2016-09-19
- Assignees: Gree Electric Appliances, Inc. of Zhuhai
- Applicants: Gree Electric Appliances, Inc. of Zhuhai
- Inventors: Guangyou Liu; Wencheng Zheng; Yuehui Mao; Zi Wang; Yao Chen; Bo Liang
- Web page: https://patentgrep.com/patent/US20210280179A1

## CPC

- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- F24F 11/30: for purposes related to the operation of the system, e.g. for safety or monitoring
- F24F 11/50: characterised by user interfaces or communication
- F24F 11/62: characterised by the type of control or by internal processing, e.g. using fuzzy logic, adaptive control or estimation of values
- F24F 2120/10: Occupancy
- G10L 15/25: using position of the lips, movement of the lips or face analysis
- H04L 12/2816: {Controlling appliance services of a home automation network by calling their functionalities (arrangements in telecontrol or telemetry systems for selectively calling a substation from a main station; in which substation desired apparatus is selected for applying a control signal thereto or for obtaining measured values therefrom H04Q9/00)}
- F24F 11/61: using timers
- F24F 2120/12: Position of occupants
- H04R 3/005: {for combining the signals of two or more microphones (specially adapted for hearing aids H04R25/407)}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- F24F: AIR-CONDITIONING; AIR-HUMIDIFICATION; VENTILATION; USE OF AIR CURRENTS FOR SCREENING (removing dirt or fumes from areas where they are produced B08B15/00; vertical ducts for carrying away waste gases from buildings E04F17/02; tops for chimneys or ventilating shafts, terminals for flues F23L17/02)
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K)

## Abstract

A household appliance control method, device and system, and an intelligent air conditioner are provided. The method includes that: a sound source location is determined by means of a camera; voice information of a user is picked up according to the sound source location; and at least one control operation is performed on a household appliance according to the voice information. The voice information matches at least one corresponding control instruction. By means of the method, a location of at least one user can be detected according to the camera, thereby enhancing at least one pickup audio signal at the location of the at least one user.

## Claims

### Claim 1

A household appliance control method, comprising:
determining a sound source location by means of a camera;
picking up a voice information of a user according to the sound source location; and
performing at least one control operation on a household appliance according to the voice information, wherein the voice information matches at least one corresponding control instruction.

### Claim 2 (depends on 1)

The method as claimed in claim 1, wherein determining the sound source location by means of the camera comprises:
acquiring image information in a household appliance scenario by means of the camera;
analyzing the image information to recognize user information in the household appliance scenario; and
determining the sound source location according to the user information.

### Claim 3 (depends on 2)

The method as claimed in claim 2, wherein the user information comprises at least one of the following: the number of at least one user, a location of at least one user, and a mouth shape of at least one user;
determining the sound source location according to the user information comprises:
when the number of the at least one user represents that there is one user, determining the location of this one user as the sound source location; and
when the number of the at least one represents that there are a plurality of users, determining one user who produces a correct voice according to the mouth shape of the at least one user, and determining the location of this one user as the sound source location.

### Claim 4 (depends on 1)

The method as claimed in claim 1, wherein picking up the voice information of the user according to the sound source location comprises:
enhancing at least one pickup audio signal of an array type dual microphone at the sound source location; and
picking up the voice information of the user according to the at least one pickup audio signal.

### Claim 5 (depends on 1)

The method as claimed in claim 1, before determining the sound source location by means of the camera, the method further comprising:
acquiring wake-up operation information of the user, wherein the wake-up operation information comprises at least one of the following: gesture information and voice information; and
triggering the household appliance to switch from a standby state to a working state according to the wake-up operation information.

### Claim 6 (depends on 1)

The method as claimed in claim 1, the method further comprising:
acquiring gesture information of the user; and
performing the at least one control operation on the household appliance according to the gesture information.

### Claim 7 (depends on 6)

The method as claimed in claim 6, wherein after the household appliance enters into the working state, when neither voice information is picked up within a preset time period nor gesture information is acquired within the preset time period, the household appliance is triggered to re-enter into the standby state.

### Claim 8 (depends on 5)

The method as claimed in claim 5, after triggering the household appliance to enter into the working state according to the wake-up operation information, the method further comprising:
acquiring end operation information from the user, wherein the end operation information comprises at least one of the following: gesture information and voice information; and
triggering the household appliance to exit the working state according to the end operation information.

### Claim 9

A household appliance control device, comprising:
a positioning component, configured to determine a sound source location by means of a camera;
a pickup component, configured to pick up voice information of a user according to the sound source location; and
a control component, configured to perform at least one control operation on a household appliance according to the voice information, wherein the voice information matches at least one corresponding control instruction.

### Claim 10 (depends on 9)

The device as claimed in claim 9, wherein the positioning component comprises:
an image acquisition element, configured to acquire image information in a household appliance scenario by means of the camera;
an information recognition element, configured to analyze the image information to recognize user information in the household appliance scenario; and
a positioning element, configured to determine the sound source location according to the user information.

### Claim 11 (depends on 10)

The device as claimed in claim 10, wherein the user information comprises at least one of the following: the number of at least one user, a location of at least one user, and a mouth shape of at least one user;
the positioning element is configured to determine, when the number of the at least one user represents that there is one user, the location of this one user as the sound source location, and determine, when the number of the at least one represents that there are a plurality of users, one user who produces a correct voice according to the mouth shape of the at least one user, and determining the location of this one user as the sound source location.

### Claim 12 (depends on 9)

The device as claimed in claim 9, wherein the pickup component comprises:
an enhancement element, configured to enhance at least one pickup audio signal of an array type dual microphone at the sound source location; and
a pickup element, configured to pick up the voice information of the user according to the at least one pickup audio signal.

### Claim 13 (depends on 9)

The device as claimed in claim 9, further comprising:
a wake-up component, configured to acquire wake-up operation information of the user, wherein the wake-up operation information comprises at least one of the following: gesture information and voice information; and trigger the household appliance to switch from a standby state to a working state according to the wake-up operation information.

### Claim 14 (depends on 13)

The device as claimed in claim 13, further comprising:
an exit component, configured to acquire end operation information from the user, wherein the end operation information comprises at least one of the following: gesture information and voice information; and trigger the household appliance to exit the working state according to the end operation information.

### Claim 15 (depends on 9)

An intelligent air conditioner, comprising the household appliance control device as claimed in claim 9.

### Claim 16

A household appliance control system, comprising:
a camera, configured to capture image information in a household appliance scenario;
an image recognition component, configured to extract the image information, and analyze the image information to determine a sound source location;
a microphone, configured to pick up voice information of a user according to the sound source location;
a voice recognition component, configured to analyze the voice information to determine at least one control instruction corresponding to the voice information; and
a main control component, configured to perform at least one control operation on a household appliance according to the at least one control instruction.

### Claim 17 (depends on 16)

The system as claimed in claim 16, wherein
the main control component is further configured to control the microphone to enhance at least one pickup audio signal at the sound source location; and
the microphone is further configured to pick up the voice information of the user according to the at least one enhanced pickup audio signal.

### Claim 18 (depends on 16)

The system as claimed in claim 16, further comprising:
a WIFI component, connected with a client to achieve interaction between the client and the main control component;
the client, configured to interact with the main control component; and
a voice playing component, configured to broadcast voice information to the user to achieve interaction with the user.
