Back to results
US 11,539,546 B2Granted
Home appliances and method for controlling home appliances
AssigneeLG Electronics Inc.
InventorYun Sik Park
Life of this patent
Estimating the term…
Abstract
A method of controlling a home appliance which operates in an Internet of Things environment through a 5G communication network and which is performed using a neural network model generated by machine learning, including determining whether there is a user in the vicinity of the home appliance, capturing a motion of the user using a vision sensor based on a determination that there is a user in the vicinity of the home appliance, identifying an intention of the user based on the captured motion, and activating a speech module of the home appliance based on the intention of the user.
Claims17
A method of controlling a home appliance, the home appliance including:
- a camera;
- a speech engine processor for natural language processing;
- a speaker;
- a microphone;
- a memory;
- and
- a controller including a processor,
- the method comprising:
- monitoring, via the camera, a predetermined area around the home appliance;
- determining, via the controller, whether there is a user in the predetermined area around the home appliance based on the monitoring of the predetermined area by the camera;
- capturing, via the camera, motion of the user, when the controller determines that the user is in the predetermined area around the home appliance;
- identifying, via the controller, an intention of the user based on the captured motion;
- and
- activating the speech engine processor based on the intention of the user,
- wherein the identifying the intention of the user includes using a neural network model trained to determine whether the user intends to manipulate the home appliance based on the motion, captured by the camera, of the user in the predetermined area around the home appliance, and
- wherein the method further comprises:
- after the activating the speech engine processor of the home appliance, determining, via the controller, that the identifying of the intention of the user using the neural network model has failed based on a command not being received by the home appliance for a predetermined time, labeling an image of the captured motion as training data in which the user does not intend to use the home appliance and updating the neural network model using the labeled image of the captured motion.
14 more elements
A home appliance, comprising:
- a cabinet forming an exterior of the home appliance;
- a door configured to open or close a passage through which an object is to be loaded into the cabinet;
- a camera configured to capture an image of outside of the home appliance;
- a speech engine processor;
- a speaker configured to output a voice signal generated in the speech engine processor;
- a controller configured to control the home appliance, the controller including a processor;
- and
- a memory connected to the controller, wherein the memory stores instructions configured to, when executed by the controller, cause the controller to:
- determine whether there is a user in a predetermined area around the home appliance,
- capture, via the camera, a motion of the user when the controller determines that the user is in the predetermined area around the home appliance,
- identify an intention of the user based on the captured motion, and
- activate the speech engine processor in accordance with the intention of the user,
- wherein the memory stores a neural network model trained to determine whether the user intends to manipulate the home appliance based on the motion of the user in the predetermined area around the home appliance,
- wherein identifying the intention of the user is performed using the neural network model, and
- wherein the memory further stores instructions configured to, after the activating the speech engine processor of the home appliance, determine, via the controller, that the identifying of the intention of the user using the neural network model has failed based on a command not being received by the home appliance for a predetermined time, label an image of the captured motion as training data in which the user does not intend to use the home appliance and update the neural network model using the labeled image of the captured motion.
12 more elements
Dependencies are inferred from the claim text.