# US 2014/0191954 A1 · Gesture Based User Interface Supporting Preexisting Symbols > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2014/0191954 A1 - Document ID: US20140191954A1 - Status: Published application - Kind: A1 - Application number: 14/204,320 - Filed: 2014-03-11 - Published: 2014-07-10 - Earliest claimed: 2004-03-23 - Assignees: Fujitsu Limited - Applicants: Fujitsu Limited - Inventors: David L. Marvit; Hitoshi Matsumoto; Albert H.M. Reinhardt - Web page: https://patentgrep.com/patent/US20140191954A1 ## CPC - G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)} - G06F 1/1613: {for portable computers (cooling arrangements therefor G06F1/203; constructional details or arrangements for pocket calculators, electronic agendas or books G06F15/0216; constructional details of portable telephone sets: with several bodies H04M1/0202)} - G06F 1/1686: {the I/O peripheral being an integrated camera} - G06F 1/1694: {the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer} - G06F 3/0236: {using selection techniques to select from displayed items} - G06F 3/0346: with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors - G06F 2200/1637: Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A motion controlled handheld device includes a display having a viewable surface and operable to generate an image and a gesture database maintaining a plurality of gestures. Each gesture is defined by a motion of the device with respect to a first position of the device. The gestures comprise symbol gestures each corresponding to a character from a preexisting character set. The device includes an application database maintaining at least one application and a gesture mapping database comprising a gesture input map for the application. The gesture input map comprises mappings of the symbol gestures to corresponding inputs for the application. The device includes a motion detection module operable to detect motion of the handheld device within three dimensions and to identify components of the motion in relation to the viewable surface. The device also includes a control module operable to load the application, to track movement of the handheld device using the motion detection module, to compare the tracked movement against the symbol gestures to identify a matching symbol gesture, to identify, using the gesture input map, the corresponding input mapped to the matching symbol gesture, and to provide the corresponding input to the application. ## Claims ### Claim at position 1 1-21. (canceled) ### Claim 22 A handheld motion control system comprising: a motion detector having a three-axis acceleration sensor; a device locator configured to identify and select a device to be controlled from among a plurality of devices based on a direction of movement of the motion detector; and a wireless communication interface configured to communicate information obtained by the motion detector to a processing apparatus of the selected device configured to obtain the information from the wireless communication interface to process the information. ### Claim 23 (depends on 22) The motion control system according to claim 22, wherein the handheld device further comprises a camera with an axis of the camera along a third axis perpendicular to each of a first axis and a second axis of the three-axis acceleration sensor. ### Claim 24 (depends on 22) The motion control system according to claim 22, wherein the handheld device further comprises a camera with an axis of the camera in a direction along a longitudinal side of the handheld device. ### Claim 25 (depends on 22) The motion control system according to claim 22, wherein the handheld device further comprises a camera with an axis of the camera of the handheld device along an axis that is parallel to a mounting surface of the three-axis acceleration sensor. ### Claim 26 (depends on 22) The motion control system according to claim 22, wherein the motion detector of the handheld device includes a processor operable to allow a user to repeatedly selectively engage and disengage motion sensitivity. ### Claim 27 (depends on 22) The motion control system according to claim 22, wherein the handheld device is operable to provide feedback information to notify a user that a particular motion of the handheld device is recognized as a particular gesture. ### Claim 28 (depends on 27) The motion control system according to claim 27, wherein the handheld device is operable to provide the feedback information in an audio format. ### Claim 29 (depends on 27) The motion control system according to claim 27, wherein the handheld device is operable to provide the feedback information in a vibratory format. ### Claim 30 (depends on 22) The motion control system according to claim 22, wherein the processing apparatus is a component of a DVD player. ### Claim 31 (depends on 22) The motion control system according to claim 22, wherein the processing apparatus is a component of a television. ### Claim 32 (depends on 22) The motion control system according to claim 22, wherein the handheld device is connected to other computer peripheral equipment. ### Claim 33 (depends on 22) The motion control system according to claim 22, wherein the handheld device further comprises a unit that notifies a user of output. ### Claim 34 (depends on 22) The motion control system according to claim 22, wherein the handheld device further comprises a unit that notifies a user of a battery level. ### Claim 35 (depends on 22) The motion control system according to claim 22, wherein the handheld device further comprises a unit that notifies via a sound. ### Claim 36 (depends on 22) The motion control system according to claim 22, wherein the handheld device further comprises a user interface. ### Claim 37 (depends on 22) The motion control system according to claim 22, wherein the handheld device further comprises a unit that gives a vibratory feedback. ### Claim 38 (depends on 23) The motion control system according to claim 23, wherein the camera is configured to provide information used for a distance measurement. ### Claim 39 (depends on 24) The motion control system according to claim 24, wherein the camera is configured to provide information used for a distance measurement. ### Claim 40 (depends on 25) The motion control system according to claim 25, wherein the camera is configured to provide information used for a distance measurement. ### Claim 41 (depends on 22) The motion control system according to claim 22, wherein the processing apparatus comprises a calendar application. ### Claim 42 (depends on 22) The motion control system according to claim 22, wherein the motion detector is operable to allow a user to reset a base reference position of the handheld device. ### Claim 43 (depends on 22) The motion control system according to claim 22, wherein the processing apparatus is operable to provide electronic mail functionality. ### Claim 44 (depends on 22) The motion control system according to claim 22, wherein the processing apparatus is operable to provide a navigation menu. ### Claim 45 (depends on 22) The motion control system according to claim 22, wherein the processing apparatus is operable to provide time and date information. ### Claim 46 (depends on 22) The motion control system according to claim 22, wherein the processing apparatus further comprises a gesture mapping database. ### Claim 47 (depends on 22) The motion control system according to claim 22, wherein the processing apparatus further comprises a gesture database. ### Claim 48 (depends on 47) The motion control system according to claim 47, wherein the gesture database is operable to record a gesture made by a user. ### Claim 49 (depends on 22) The motion control system according to claim 22, wherein: the handheld device further comprises a camera; and the motion detector is operable to detect a tilt and translation motion based on information obtained by the three-axis acceleration sensor and the camera. ### Claim 50 A method for controlling a motion control system including a handheld device and a processing apparatus, the method comprising: detecting movement of the handheld device by using a motion detector having a three-axis acceleration sensor; identifying and selecting a device to be controlled from among a plurality of devices based on the detected movement of the handheld device; communicating the movement obtained by the motion detector to the processing apparatus through a wireless communication interface of the identified and selected device for processing of the obtained movement by the processing apparatus.