# US 2008/0259055 A1 · Manipulating An Object Utilizing A Pointing Device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2008/0259055 A1
- Document ID: US20080259055A1
- Status: Published application
- Kind: A1
- Application number: 12/104,360
- Filed: 2008-04-16
- Published: 2008-10-23
- Earliest claimed: 2002-02-07
- Assignees: Microsoft Corporation
- Inventors: Andrew Wilson
- Web page: https://patentgrep.com/patent/US20080259055A1

## CPC

- G06F 3/0346: with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
- G06F 3/038: Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
- G08C 17/00: Arrangements for transmitting signals characterised by the use of a wireless electrical link
- G06F 2203/0381: Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
- G08C 2201/31: Voice input
- G08C 2201/32: Remote control based on movements, attitude of remote control device
- G08C 2201/41: Remote control of gateways
- G08C 2201/50: Receiving or transmitting feedback, e.g. replies, status updates, acknowledgements, from the controlled devices
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G08C: TRANSMISSION SYSTEMS FOR MEASURED VALUES, CONTROL OR SIMILAR SIGNALS (fluid pressure transmitting systems F15B; mechanical means for transferring the output of a sensing member into a different variable G01D5/00; mechanical control systems G05G)

## Abstract

The present invention is directed toward a system and process that controls a group of networked electronic components using a multimodal integration scheme in which inputs from a speech recognition subsystem, gesture recognition subsystem employing a wireless pointing device and pointing analysis subsystem also employing the pointing device, are combined to determine what component a user wants to control and what control action is desired. In this multimodal integration scheme, the desired action concerning an electronic component is decomposed into a command and a referent pair. The referent can be identified using the pointing device to identify the component by pointing at the component or an object associated with it, by using speech recognition, or both. The command may be specified by pressing a button on the pointing device, by a gesture performed with the pointing device, by a speech recognition event, or by any combination of these inputs.

## Claims

### Claim 1

A wireless, hand-held pointing device utilized to manipulate an object displayed on a display screen, the pointing device comprising:
a case extending in a longitudinal direction;
at least one button positioned on a top surface of the case;
a power supply within the case to provide power to the pointing device;
an accelerometer within the case; and
a controller within the case configured to receive power from the power supply, to sense an activation or a deactivation of the button, and to process output from the accelerometer in order to detect a motion of the pointing device, wherein the object displayed on the display screen is manipulated in accordance with the detected motion of the pointing device.

### Claim 2 (depends on 1)

The pointing device of claim 1 further comprising a component on a front surface of the case, which is coupled to the controller and utilized by the controller to obtain image data.

### Claim 3 (depends on 2)

The pointing device of claim 2, wherein a location of the pointing device is determined using the image data.

### Claim 4 (depends on 1)

The pointing device of claim 1, wherein the pointing device wirelessly transmits one or more data messages to a receiving unit remote from the pointing device.

### Claim 5 (depends on 4)

The pointing device of claim 4, wherein the one or more data messages include acceleration data detected by the accelerometer.

### Claim 6 (depends on 5)

The pointing device of claim 5, wherein the pointing device wirelessly transmits the one or more data messages periodically.

### Claim 7 (depends on 6)

The pointing device of claim 6, wherein the receiving unit demodulates one or more received signals.

### Claim 8 (depends on 1)

The pointing device of claim 1 further comprising a gyroscope within the case to detect motion of the pointing device.

### Claim 9 (depends on 1)

The pointing device of claim 1 further comprising a magnetometer within the case to detect motion of the pointing device.

### Claim 10 (depends on 1)

The pointing device of claim 1 further comprising one or more indicator lights.

### Claim 11 (depends on 1)

The pointing device of claim 1 further comprising one or more light-emitting diodes (LEDs).

### Claim 12 (depends on 11)

The pointing device of claim 11, wherein the LEDs provide an indication of status information.

### Claim 13 (depends on 1)

The pointing device of claim 1, wherein the pointing device receives one or more data messages from a remote device and decodes the data messages with the controller.

### Claim 14 (depends on 1)

The pointing device of claim 1, wherein the power supply comprises one or more batteries.

### Claim 15 (depends on 1)

The pointing device of claim 1, wherein the at least one operation button, when selected, provides an indication of a function.

### Claim 16 (depends on 1)

The pointing device of claim 1, wherein the pointing device outputs a state of the at least one operation button.

### Claim 17 (depends on 1)

The pointing device of claim 1, wherein the pointing device is associated with an identifier, which uniquely identifies the pointing device.

### Claim 18 (depends on 1)

The pointing device of claim 1, wherein information or commands are entered via the pointing device.

### Claim 19 (depends on 1)

The pointing device of claim 1, wherein the output of the accelerometer describes an acceleration of the pointing device.

### Claim 20 (depends on 1)

The pointing device of claim 1, wherein the output of the accelerometer describes acceleration of the pointing device along a x-axis, a y-axis, a z-axis, or a combination thereof.

### Claim 21 (depends on 13)

The pointing device of claim 13, wherein the one or more data messages are configured to activate a vibration unit in the case after being decoded by the controller.

### Claim 22

A method for manipulating an object on a display screen with a wireless pointing device, the wireless pointing device including a case, a button on a top of the case, a power supply in the case to supply power to the wireless pointing device, at least one sensor in the case, and a controller in the case coupled to the button, the power supply, and the at least one sensor, the method comprising:
detecting a motion of the wireless pointing device by the at least one sensor;
receiving with the controller an output from the at least one sensor associated with the motion of the wireless pointing device;
generating a signal associated with the output received by the controller;
transmitting the signal; and
manipulating an object on a display screen using the signal.

### Claim 23 (depends on 22)

The method of claim 22 further comprising detecting with the controller a state of the button, wherein the signal further comprises information associated with the state of the button.

### Claim 24 (depends on 22)

The method of claim 22, wherein the wireless pointing device further comprises a component configured to obtain image data positioned on a front of the case, the method further comprising:
receiving image data; and
determining a location of the wireless pointing device using the image data.

### Claim 25 (depends on 22)

The method of claim 22, wherein the at least one sensor comprises an accelerometer, and wherein the step of detecting further comprises detecting an acceleration of the wireless pointing device by the accelerometer.

### Claim 26 (depends on 25)

The method of claim 25, wherein the output received by the controller from the accelerometer includes acceleration data.

### Claim 27 (depends on 26)

The method of claim 26, wherein the signal generated by the controller includes the acceleration data.

### Claim 28 (depends on 22)

The method of claim 22, wherein the step of transmitting the signal further comprises transmitting the signal periodically.

### Claim 29 (depends on 22)

The method of claim 22, further comprising receiving one or more data messages from a remote device.

### Claim 30 (depends on 22)

The method of claim 22, wherein the step of detecting further comprises:
detecting a motion of the wireless pointing device along an x-axis;
detecting a motion of the wireless pointing device along a y-axis; and
detecting a motion of the wireless pointing device along a z-axis.

### Claim 31

An object manipulation system comprising:
a switching system connected to a device, the switching system configured to be activated and deactivated;
an acceleration detection system within the device, the acceleration system configured to determine an acceleration of the device in 3-dimensional space;
a control system coupled to the switching system and the acceleration system, the control system configured to receive data associated with the activation and the deactivation of the switching system and the acceleration of the device in 3-dimensional space and to generate signals associated with the received data; and
a communication system configured to transmit the signal, wherein the transmitted signal is capable of being used to manipulate an object displayed on a display screen.
