# US 9,436,290 B2 · Display device and operating method thereof > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,436,290 B2 - Document ID: US9436290B2 - Status: Granted - Kind: B2 - Application number: 14/457,951 - Filed: 2014-08-12 - Published: 2015-10-01 - Granted: 2016-09-06 - Earliest claimed: 2014-03-27 - Assignees: LG Electronics Inc. - Applicants: LG Electronics Inc. - Inventors: Sangkyu Kang; Wooseok Ahn; Eunyul Lee; Sewon Park; Dongheon Shin - Web page: https://patentgrep.com/patent/US9436290B2 ## CPC - G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)} - G06V 40/28: {Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113)} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A method of controlling a display device, and which includes obtaining, via camera of the display device, a user's image; recognizing, via a controller of the display device, a user's hand gesture from the obtained user's image; and when the recognized user's hand gesture does not match a pre-stored gesture, displaying, via a display of the display device, a function guide including different hand gestures different than the recognized user's hand gesture for guiding a user's selection for performing a function of the display device. ## Claims ### Claim 1 A method of controlling a display device, the method comprising: obtaining, via a camera of the display device, a user's image; recognizing, via a controller of the display device, a user's hand gesture from the obtained user's image; and if the recognized user's hand gesture does not match a pre-stored gesture, displaying, via a display of the display device, a function guide including different hand gestures different than the recognized user's hand gesture for guiding a user's selection for performing a function of the display device, wherein the different hand gestures include a plurality of similar hand gestures representing a gesture pattern similar to the recognized user's hand gesture, wherein the function guide comprises: an image of the recognized user's hand gesture; and a plurality of images of the different hand gestures, wherein the images of the different hand gestures are disposed around the image of the recognized user's hand gesture, wherein the method further comprises: recognizing, by using the obtained user's image, that the user's hand is moved toward a direction where one image corresponding to a different hand gesture among the plurality of images of the different hand gestures is disposed, selecting the image of a different hand gesture corresponding to the direction based on a recognition result, and performing a function of the display device corresponding to the selected image of the different hand gesture, and wherein the function of the display device represents a function related to an edition or a play of a media content. ### Claim 2 (depends on 1) The method according to claim 1, wherein each of the images of the different hand gestures is disposed at the top, bottom, left, and right of the image of the recognized user's hand gesture. ### Claim 3 (depends on 2) The method according to claim 2, wherein the function guide further comprises a plurality of function inducers disposed between the image of the recognized hand gesture and the images of each of the different hand gestures and inducing a user's selection to perform a function corresponding to each of the different hand gestures. ### Claim 4 (depends on 3) The method according to claim 3, wherein the function guide further comprises a plurality of similarity indicators representing a degree of similarity between each of the different hand gestures and the recognized user's hand gesture. ### Claim 5 (depends on 4) The method according to claim 4, further comprising: obtaining an input for selecting a specific different hand gesture from the different hand gestures; and performing a function of the display device corresponding to the selected specific different hand gesture. ### Claim 6 (depends on 5) The method according to claim 5, wherein the obtaining of the input for selecting the specific different hand gesture comprises obtaining an input for selecting the specific different hand gesture by detecting a movement direction of the user's hand. ### Claim 7 (depends on 1) The method according to claim 1, wherein the function guide comprises: images of a plurality of basic hand gestures representing a basic gesture pattern pre-specified or predetermined by the user; and text disposed at one side of each image of the basic hand gestures and representing a function of each of the plurality of basic hand gestures, and wherein each image of the basic hand gestures is disposed around the image of the recognized user's hand gesture. ### Claim 8 (depends on 1) The method according to claim 1, further comprising recognizing the user's face from the obtained user's image, wherein the displaying of the function guide comprises, when the recognized user's hand gesture is not the pre-stored gesture, displaying the function guide based on a position relationship of the recognized user's face and the recognized user's hand gesture. ### Claim 9 (depends on 8) The method according to claim 8, wherein when the recognized user's hand gesture is not the pre-stored gesture, the displaying of the function guide based on the position relationship of the recognized user's face and the recognized user's hand gesture comprises: detecting whether the recognized user's hand gesture is disposed at the right or left based on the user's face; and displaying the function guide based on a detection result, wherein the function guide comprises: images of a plurality of similar hand gestures representing a gesture pattern similar to the recognized user's hand gesture; and text disposed at one side of each image of the plurality of similar hand gestures and representing a function of each of the plurality of similar hand gestures, and wherein each image of the plurality of similar hand gestures is disposed around the recognized user's hand gesture. ### Claim 10 (depends on 8) The method according to claim 8, wherein when the recognized user's hand gesture is not the pre-stored gesture, the displaying of the function guide based on the position relationship of the recognized user's face and the recognized user's hand gesture comprises: detecting whether the recognized user's hand gesture is disposed at the right or left based on the user's face; and displaying the function guide based on a detection result, wherein the function guide comprises: images of a plurality of basic hand gestures representing a basic gesture pattern; and text disposed at one side of each of the plurality of basic hand gestures and representing a function of each of the plurality of basic hand gestures, wherein each image of the plurality of basic hand gestures is disposed around the recognized user's hand gesture. ### Claim 11 A display device, comprising: a display; a memory configured to store gestures; a camera configured to obtain a user's image; and a controller configured to: recognize a user's hand gesture from the obtained user's image, and if the recognized user's hand gesture does not match a pre-stored gesture, display on the display a function guide including different hand gestures different than the recognized user's hand gesture for guiding a user's selection for performing a function of the display device, wherein the different hand gestures include a plurality of similar hand gestures representing a gesture pattern similar to the recognized user's hand gesture, wherein the function guide comprises: an image of the recognized user's hand gesture; and a plurality of images of the different hand gestures, wherein the images of the different hand gestures are disposed around the image of the recognized user's hand gesture, wherein the controller is further configured to: recognize, by using the obtained user's image, that the user's hand is moved toward a direction where one image corresponding to a different hand gesture among the plurality of images of the different hand gestures is disposed, select the image of a different hand gesture corresponding to the direction based on a recognition result, and perform a function of the display device corresponding to the selected image of the different hand gesture, and wherein the function of the display device represents a function related to an edition or a play of a media content. ### Claim 12 (depends on 11) The display device according to claim 11, wherein each of the images of the different hand gestures is disposed at the top, bottom, left, and right of the image of the recognized user's hand gesture. ### Claim 13 (depends on 12) The display device according to claim 12, wherein the function guide further comprises a plurality of function inducers disposed between the image of the recognized hand gesture and the images of each of the different hand gestures and inducing a user's selection to perform a function corresponding to each of the different hand gestures. ### Claim 14 (depends on 13) The display device according to claim 13, wherein the function guide further comprises a plurality of similarity indicators representing a degree of similarity between each of the different hand gestures and the recognized user's hand gesture. ### Claim 15 (depends on 14) The display device according to claim 14, wherein the controller is further configured to: obtain an input for selecting a specific different hand gesture from the different hand gestures, and perform a function of the display device corresponding to the selected specific different hand gesture. ### Claim 16 (depends on 15) The display device according to claim 15, wherein the controller is further configured to obtain the input for selecting the specific different hand gesture by detecting a movement direction of the user's hand. ### Claim 17 (depends on 11) The display device according to claim 11, wherein the function guide comprises: images of a plurality of basic hand gestures representing a basic gesture pattern pre-specified or predetermined by the user; and text disposed at one side of each image of the basic hand gestures and representing a function of each of the plurality of basic hand gestures, and wherein each image of the basic hand gestures is disposed around the image of the recognized user's hand gesture. ### Claim 18 (depends on 11) The display device according to claim 11, wherein the controller is further configured to: recognize the user's face from the obtained user's image, and when the recognized user's hand gesture is not the pre-stored gesture, display on the display the function guide based on a position relationship of the recognized user's face and the recognized user's hand gesture. ### Claim 19 (depends on 1) The method according to claim 1, wherein the plurality of similar hand gestures are displayed on the function guide if a similarity between the recognized gesture and each of a plurality of gestures stored in the memory is less than a reference similarity. ### Claim 20 (depends on 11) The display device according to claim 11, wherein the plurality of similar hand gestures are displayed on the function guide if a similarity between the recognized gesture and each of a plurality of gestures stored in the memory is less than a reference similarity.