# US 2014/0157206 A1 · Mobile device providing 3d interface and gesture controlling method thereof > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2014/0157206 A1 - Document ID: US20140157206A1 - Status: Published application - Kind: A1 - Application number: 13/828,576 - Filed: 2013-03-14 - Published: 2014-06-05 - Earliest claimed: 2012-11-30 - Assignees: Samsung Electronics Co., Ltd. - Applicants: Samsung Electronics Co., Ltd. - Inventors: Ilia OVSIANNIKOV; Dong-Ki MIN; Yoon-dong PARK - Web page: https://patentgrep.com/patent/US20140157206A1 ## CPC - G06F 3/0485: Scrolling or panning - G06F 1/1686: {the I/O peripheral being an integrated camera} - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)} - G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)} - G06F 3/0304: {Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042)} - G06F 3/04815: Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object - G06F 3/04845: for image manipulation, e.g. dragging, rotation, expansion or change of colour - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Disclosed is a gesture control method for a mobile device that provides a three-dimensional interface. The gesture control method includes displaying a virtual three-dimensional space using the three-dimensional interface; detecting at least one gesture of at least one user using at least one front-facing sensor; and moving an object existing in the virtual three-dimensional space according to the detected gesture such that the at least one user interacts with the virtual three-dimensional space. ## Claims ### Claim 1 A gesture control method for a mobile device that provides a three-dimensional interface, comprising: displaying a virtual three-dimensional space using the three-dimensional interface; detecting at least one gesture of at least one user using at least one front-facing sensor; and moving an object existing in the virtual three-dimensional space according to the detected gesture such that the at least one user interacts with the virtual three-dimensional space. ### Claim 2 (depends on 1) The gesture control method of claim 1, further comprising: generating an avatar corresponding to a hand of the at least one user based on location information of the at least one user. ### Claim 3 (depends on 1) The gesture control method of claim 1, further comprising: displaying a three-dimensional scene corresponding to a still space in the virtual three-dimensional space, such that the at least one user is immersed in the three-dimensional space, the still space associated with a peripheral circumstance of the at least one user; displaying the three-dimensional scene as if a cube is floated in the still space; and changing an appearance of the cube displayed according to a motion of the mobile device or the at least one user when the mobile device or the at least one user moves, such that a location of the cube is not moved within the three-dimensional space. ### Claim 4 (depends on 3) The gesture control method of claim 3, wherein the changing an appearance of the cube comprises: displaying a left side of the cube more compared with the appearance of the cube before the at least one user moves if a head of the at least one user moves leftward. ### Claim 5 (depends on 1) The gesture control method of claim 1, further comprising: acquiring and tracing coordinates of eyes of the at least one user within a physical three-dimensional space using the at least one front-facing sensor. ### Claim 6 (depends on 5) The gesture control method of claim 5, further comprising: varying the virtual three-dimensional space according to the coordinates of the eyes such that the at least one user is immersed in the virtual three-dimensional space. ### Claim 7 (depends on 1) The gesture control method of claim 1, further comprising: displaying the virtual three-dimensional space to superimpose the virtual three-dimensional space on a physical scene that the at least one users watches. ### Claim 8 (depends on 1) The gesture control method of claim 1, further comprising: generating an avatar of the at least one user in the virtual three-dimensional space; and communicating with another user other than the at least one user using the generated avatar. ### Claim 9 (depends on 1) The gesture control method of claim 1, further comprising: selecting an object of the virtual three-dimensional space based on pinching by the at least one user. ### Claim 10 (depends on 9) The gesture control method of claim 9, further comprising: entering a resizing mode for resizing the object selected based on squeezing by the at least one user. ### Claim 11 (depends on 10) The gesture control method of claim 10, further comprising: terminating the resizing mode when the selected object is not resized during a desired time. ### Claim 12 (depends on 9) The gesture control method of claim 9, further comprising: moving the object based on pushing the object with at least one of a hand of the at least one user and a hand of an avatar corresponding to the hand of the at least one user. ### Claim 13 (depends on 9) The gesture control method of claim 9, further comprising: panning the object based on rotating the object with at least one of a hand of the at least one user and a hand of an avatar corresponding to the hand of the at least one user. ### Claim 14 A mobile device that provides a three-dimensional interface, comprising: a communication unit configured to perform wireless communication; a memory unit configured to store user data and data; a display unit configured to display a virtual three-dimensional space using the three-dimensional interface; a sensing unit configured to sense a still picture and a moving picture of a physical space and including at least one front-facing sensor configured to sense at least one gesture of a user; and at least one processor configured to control the communication unit, the memory unit, the display unit, and the sensing unit, wherein the at least one processor moves an object existing in the virtual three-dimensional space according to the detected gesture, and when the mobile device or the user moves, the at least one processor controls the three-dimensional interface such that a sight of the user toward the object is varied. ### Claim 15 (depends on 14) The mobile device of claim 14, wherein the front-facing sensor is a time-of-flight camera. ### Claim 16 A gesture control method of a mobile device configured to provide a three-dimensional interface, the method comprising: displaying a first portion of a virtual three-dimensional space so that the first portion is superimposed on a physical space, using the three dimensional interface; displaying a second portion of the virtual three-dimensional space using the three-dimensional interface; detecting at least one gesture of at least one user using at least one of one or more front-facing sensors and one or more back-facing sensors; and moving an object existing in the virtual three-dimensional space according to the detected gesture such that the at least one user interacts with the virtual three-dimensional space. ### Claim 17 (depends on 16) The gesture control method of claim 16, wherein the second portion of the virtual three-dimensional space includes a floating cube. ### Claim 18 (depends on 17) The gesture control method of claim 17, further comprising: changing an appearance of the cube according to a motion of the mobile device or the at least one user when the mobile device or the at least one user moves, such that a location of the cube is not moved within the three-dimensional space. ### Claim 19 (depends on 16) The gesture control method of claim 16, further comprising: generating a hand of an avatar corresponding to a hand of the at least one user based on location information of the at least one user. ### Claim 20 (depends on 19) The gesture control method of claim 19, further comprising: interacting with an object in the virtual three-dimensional space based on movement of the hand of the avatar.