Back to results
US 11,887,330 B2Granted
Method and apparatus for assisting object registration in a smart home system
AssigneeKorea Advanced Institute of Science and Technology
InventorsJunyong Noh, Hyunggoog Seo, Jaedong Kim, Kwanggyoon Seo, Bumki Kim
Life of this patent
Estimating the term…
Abstract
A method of assisting object registration in a smart home system including a plurality of objects is disclosed. The disclosed method may include allowing a user to select an object from among a plurality of objects, capturing a plurality of pointing gestures by allowing the user to perform the plurality of pointing gestures at the selected object, configuring a pointing ray using each of the plurality of captured pointing gestures, estimating a position of a nearest point extending from the plurality of pointing rays, and registering the estimated position of the nearest point as a position of the selected object.
Claims11
A method of assisting object registration in a smart home system comprising a plurality of home devices, performed by a computing device, the method comprising:
- controlling, by the computing device, a user terminal to display a first screen that requests a user to select one home device from the plurality of home devices;
- receiving, by the computing device, from the user terminal, a selection of the home device;
- controlling, by the computing device, the user terminal to display a second screen that requests the user to perform a plurality of pointing gestures toward the selected home device;
- capturing, by a camera, in response to performing the plurality of pointing gestures by the user within a field of view (FOV) of the camera, the plurality of pointing gestures;
- configuring a pointing ray using each of the plurality of captured pointing gestures;
- estimating a position of a nearest point extending from the plurality of pointing rays;
- and
- registering the estimated position of the nearest point as a position of the selected home device,
- wherein the configuring of the pointing ray using each of the plurality of captured pointing gestures comprises:
- identifying a head position of the user from a corresponding captured gesture;
- identifying a hand position of the user from a corresponding captured gesture;
- defining a gesture-direction vector using the identified head position of the user and the identified hand position of the user;
- and
- configuring the pointing ray using the head position of the user and the gesture-direction vector, and
- wherein the configuring of the pointing ray using the head position of the user and the gesture-direction vector comprises configuring the pointing ray using the following equation:
- rn=Pn+an·un
- where n denotes an index indicating a pointing ray number, rn denotes an n-th pointing ray, Pn denotes a head position of the user used to configure the n-th pointing ray, an denotes a scalar parameter used to configure the n-th pointing ray, and un denotes a gesture-direction vector used to configure the n-th pointing ray.
14 more elements
A method of assisting object registration in a smart home system comprising a plurality of home devices, performed by a computing device, the method comprising:
- controlling, by the computing device, a user terminal to display a first screen that requests a user to select one home device from the plurality of home devices;
- receiving, by the computing device, from the user terminal, a selection of the home device;
- controlling, by the computing device, the user terminal to display a second screen that requests the user to perform a plurality of pointing gestures toward the selected home device;
- capturing, by a camera, in response to performing the plurality of pointing gestures by the user within a field of view (FOV) of the camera, the plurality of pointing gestures;
- configuring a pointing ray using each of the plurality of captured pointing gestures;
- estimating a position of a nearest point extending from the plurality of pointing rays;
- and
- registering the estimated position of the nearest point as a position of the selected home device,
- wherein the capturing of the plurality of pointing gestures comprises providing a plurality of images by capturing each of the plurality of pointing gestures a plurality of times,
- wherein the configuring of the pointing ray using each of the plurality of captured pointing gestures comprises:
- identifying a head position of the user a plurality of times using the plurality of images;
- calculating a mean head position of the user by averaging the identified head positions of the user;
- identifying a hand position of the user a plurality of times using the plurality of images;
- calculating a mean hand position of the user by averaging the identified hand positions of the user;
- defining a gesture-direction vector using the mean head position of the user and the mean hand position of the user;
- and
- configuring the pointing ray using the mean head position of the user and the gesture-direction vector, and
- wherein the configuring of the pointing ray using the mean head position of the user and the gesture-direction vector comprises configuring the pointing ray using the following equation:
- rn=Pn+an·un
- where n denotes an index indicating a pointing ray number, rn denotes an n-th pointing ray, Pn denotes a mean head position of the user used to configure the n-th pointing ray, an denotes a scalar value used to configure the n-th pointing ray, and un denotes a gesture-direction vector used to configure the n-th pointing ray.
17 more elements
A device for assisting object registration in a smart home system comprising a plurality of home devices, the device comprising:
- a controller;
- and
- a communicator coupled to the controller and configured to communicate with a user terminal,
- wherein the controller is configured to:
- control the user terminal to display a first screen that requests a user to select one home device from the plurality of home devices;
- receive, from the user terminal, a selection of the home device;
- control the user terminal to display a second screen that requests the user to perform a plurality of pointing gestures toward the selected home device;
- control a camera coupled to the device, to capture, in response to performing the plurality of pointing gestures by the user within a field of view (FOV) of the camera, the plurality of pointing gestures;
- configure a pointing ray using each of the plurality of captured pointing gestures;
- estimate a position of a nearest point extending from the plurality of pointing rays;
- and
- register the estimated position of the nearest point as a position of the selected object,
- wherein the controller is further configured to provide a plurality of images by capturing each of the plurality of pointing gestures a plurality of times,
- wherein the controller is further configured to identify a head position of the user a plurality of times using the plurality of images, to calculate a mean head position of the user by averaging the identified head positions of the user, to identify a hand position of the user a plurality of times using the plurality of images, to calculate a mean hand position of the user by averaging the identified hand positions of the user, to define a gesture-direction vector using the mean head position of the user and the mean hand position of the user, and to configure the pointing ray using the mean head position of the user and the gesture-direction vector, and
- wherein the controller is further configured to configure the pointing ray using the following equation:
- rn=Pn+an·un
- where n denotes an index indicating a pointing ray number, rn denotes an n-th pointing ray, Pn denotes a mean head position of the user used to configure the n-th pointing ray, an denotes a scalar value used to configure the n-th pointing ray, and un denotes a gesture-direction vector used to configure the n-th pointing ray.
14 more elements
Dependencies are inferred from the claim text.