# US 8,150,384 B2 · Methods and apparatuses for gesture based remote control

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 8,150,384 B2
- Document ID: US8150384B2
- Status: Granted
- Kind: B2
- Application number: 12/816,885
- Filed: 2010-06-16
- Published: 2011-12-22
- Granted: 2012-04-03
- Earliest claimed: 2010-06-16
- Assignees: QUALCOMM Incorporated
- Inventors: Chadia Abifaker; Ruben M. Velarde
- Web page: https://patentgrep.com/patent/US8150384B2

## CPC

- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/0346: with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
- H04L 67/125: involving control of end-device applications over a network
- H04M 1/72415: for remote control of appliances
- H04W 4/026: {using orientation information, e.g. compass}
- H04W 4/21: for social networking applications
- H04W 4/50: Service provisioning or reconfiguring
- G08C 2201/20: Binding and programming of remote control devices
- G08C 2201/32: Remote control based on movements, attitude of remote control device
- G08C 2201/71: Directional beams
- G08C 2201/91: Remote control based on location and proximity
- G08C 2201/93: Remote control using other portable devices, e.g. mobile phone, PDA, laptop
- H04M 2250/12: including a sensor for measuring a physical value, e.g. temperature or motion
- H04W 4/80: Services using short range communication, e.g. near-field communication [NFC], radio-frequency identification [RFID] or low energy communication
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- H04M: TELEPHONIC COMMUNICATION (circuits for controlling other apparatus via a telephone cable and not involving telephone switching apparatus G08)
- H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72)
- G08C: TRANSMISSION SYSTEMS FOR MEASURED VALUES, CONTROL OR SIMILAR SIGNALS (fluid pressure transmitting systems F15B; mechanical means for transferring the output of a sensing member into a different variable G01D5/00; mechanical control systems G05G)

## Abstract

Methods and apparatuses are provided that may be implemented in and/or with a mobile device to allow gesture based remote control of one or more controllable devices.

## Claims

### Claim 1

A method comprising:
estimating a location of a mobile device based, at least in part, on one or more received signals;
associating an orientation of a mobile device with a direction toward a location of a controllable device, said direction being determined based, at least in part, on said estimated location of said mobile device and said orientation of said mobile device;
indicating a selection of said controllable device;
determining one or more commands associated with said selected controllable device based, at least in part, on user input obtained via one or more inertial sensors of said mobile device; and
initiating transmission of said commands to said selected controllable device.

### Claim 2 (depends on 1)

The method as recited in claim 1, and further comprising:
determining said orientation based, at least in part, on signals associated with at least one of: said one or more inertial sensors, a location service receiver, a compass, and/or a magnetometer.

### Claim 3 (depends on 2)

The method as recited in claim 2, wherein said one or more inertial sensors comprises at least one at least one of: an accelerometer, a gyroscope, and/or a barometer.

### Claim 4 (depends on 1)

The method as recited in claim 1, wherein said one or more signals are received with a location service receiver.

### Claim 5 (depends on 1)

The method as recited in claim 1, wherein said direction comprises a directional region of space and further comprising:
indicating that said location of said controllable device is within said directional region of space.

### Claim 6 (depends on 1)

The method as recited in claim 1, and further comprising:
displaying information identifying at least said controllable device; and
obtaining user input associated with said selection of said controllable device via at least one of: an interactive or touch-screen display, a microphone, a camera, a button or keypad, and/or at least one of said one or more inertial sensors.

### Claim 7 (depends on 6)

The method as recited in claim 6, and further comprising:
displaying information identifying at least one other selectable controllable device.

### Claim 8 (depends on 6)

The method as recited in claim 6, and further comprising:
authenticating that a user may select and/or control said controllable device based, at least in part, on said user input.

### Claim 9 (depends on 1)

The method as recited in claim 1, and further comprising:
indicating said selection of said controllable device via at least one of a visual interface, an audio interface, and/or a tactile interface.

### Claim 10 (depends on 1)

The method as recited in claim 1, wherein said one or more commands identify at least one of: at least one physical act performed by a person, and/or at least one physical gesture performed using said mobile device, as determined based, at least in part, on one or more signals associated with said one or more inertial sensors.

### Claim 11 (depends on 10)

The method as recited in claim 10, wherein said at least one physical act results in at least one of an identifiable audio waveform and/or an identifiable visual image.

### Claim 12 (depends on 11)

The method as recited in claim 11, wherein said at least one physical act is identified based, at least in part, on signals associated with at least one of: a microphone and/or a camera of said mobile device.

### Claim 13 (depends on 1)

The method as recited in claim 1, and further comprising:
obtaining said one or more commands associated with said selected controllable device from at least one of: previously stored data, said controllable device, and/or from one or more other devices.

### Claim 14 (depends on 1)

The method as recited in claim 1, and further comprising:
obtaining status information associated with said selected controllable device via a network interface; and
presenting feedback to a user based, at least in part, on said status information.

### Claim 15 (depends on 1)

The method as recited in claim 1, and further comprising at least one of:
transmitting one or more signals representing said commands to at least one other device via a network interface, and/or to said selected controllable device via a network interface and/or a direct communication link.

### Claim 16

A method comprising:
with a mobile device:
obtaining an estimated location of a controllable device;
processing signals from one or more inertial sensors of said mobile device to characterize one or more gestures;
programming said mobile device to initiate transmission of one or more commands to said controllable device based, at least in part, on said characterized one or more gestures; and
programming said mobile device to select said controllable device for receiving said one or more commands responsive to an orientation of said mobile device with a direction toward said estimated location of said controllable device, said direction being determined based, at least in part, on an estimated location of said mobile device and said orientation of said mobile device, wherein said estimated location of said mobile device is based, at least in part, on one or more received signals.

### Claim 17 (depends on 16)

The method as recited in claim 16, and further comprising:
receiving signals which characterize at least one additional gesture of said one or more gestures.

### Claim 18 (depends on 16)

The method as recited in claim 16, and further comprising:
obtaining at least one of: operational information associated with said controllable device; access control information associated with controllable device; and/or identification information associated with said controllable device.

### Claim 19 (depends on 16)

The method as recited in claim 16, and further comprising:
via a camera of said mobile device, obtaining at least one of: image information associated with said controllable device; and/or identification information associated with said controllable device.

### Claim 20

An apparatus comprising:
means for estimating a location of a mobile device based, at least in part, on one or more received signals;
means for associating an orientation of a mobile device with a direction toward a location of a controllable device, said direction being determined based, at least in part, on said estimated location of said mobile device and said orientation of said mobile device;
means for indicating a selection of said controllable device;
means for determining one or more commands associated with said selected controllable device based, at least in part, on user input affecting inertial motion of said mobile device; and
means for initiating transmission of said commands to said selected controllable device.

### Claim 21 (depends on 20)

The apparatus as recited in claim 20, and further comprising:
means for determining said orientation.

### Claim 22 (depends on 20)

The apparatus as recited in claim 20, wherein said direction comprises a directional region of space and further comprising:
means for indicating that said location of said controllable device is within said directional region of space.

### Claim 23 (depends on 20)

The apparatus as recited in claim 20, and further comprising:
means for displaying information identifying at least said controllable device; and
means for obtaining user input associated with said selection of said controllable device.

### Claim 24 (depends on 23)

The apparatus as recited in claim 23, and further comprising:
means for displaying information identifying at least one other selectable controllable device.

### Claim 25 (depends on 23)

The apparatus as recited in claim 23, and further comprising:
means for authenticating that a user may select and/or control said controllable device.

### Claim 26 (depends on 20)

The apparatus as recited in claim 20, and further comprising:
means for indicating said selection of said controllable device.

### Claim 27 (depends on 20)

The apparatus as recited in claim 20, wherein said one or more commands identify at least one of: at least one physical act performed by a person, and/or at least one physical gesture performed using said mobile device.

### Claim 28 (depends on 27)

The apparatus as recited in claim 27, wherein said at least one physical act results in at least one of an identifiable audio waveform and/or an identifiable visual image.

### Claim 29 (depends on 20)

The apparatus as recited in claim 20, and further comprising:
means for obtaining said one or more commands associated with said selected controllable device from at least one of: previously stored data, said controllable device, and/or from one or more other devices.

### Claim 30 (depends on 20)

The apparatus as recited in claim 20, and further comprising:
means for obtaining status information associated with said selected controllable device via a network interface; and
means for presenting feedback to a user based, at least in part, on said status information.

### Claim 31 (depends on 20)

The apparatus as recited in claim 20, and further comprising at least one of:
means for transmitting one or more signals representing said commands to at least one other device, and/or to said selected controllable device via a direct communication link.

### Claim 32

An apparatus for use in a mobile device comprising:
means for obtaining an estimated location of a controllable device;
means for processing signals from one or more inertial sensors of said mobile device to characterize one or more gestures;
means for programming said mobile device to initiate transmission of one or more commands to said controllable device based, at least in part, on said characterized one or more gestures; and
means for programming said mobile device to select said controllable device for receiving said one or more commands responsive to an orientation of said mobile device with a direction toward said estimated location of said controllable device, said direction being determined based, at least in part, on an estimated location of said mobile device and said orientation of said mobile device, wherein said estimated location of said mobile device is based, at least in part, on one or more received signals.

### Claim 33 (depends on 32)

The apparatus as recited in claim 32, and further comprising:
means for receiving signals which characterize at least one additional gesture of said one or more gestures.

### Claim 34 (depends on 32)

The apparatus as recited in claim 32, and further comprising:
means for obtaining at least one of: operational information associated with said controllable device; access control information associated with controllable device; and/or identification information associated with said controllable device.

### Claim 35 (depends on 32)

The apparatus as recited in claim 32, and further comprising:
means for obtaining at least one of: image information associated with said controllable device; and/or identification information associated with said controllable device.

### Claim 36

A mobile device comprising:
memory;
a network interface;
a location service receiver;
one or more inertial sensors;
at least one user interface; and
at least one processing unit to: estimate a location of said mobile device based, at least in part, on one or more signals received via at least one of said network interface or said location service receiver; associate an orientation of said mobile device with a direction toward a location of a controllable device as stored in said memory, said direction being determined based, at least in part, on said estimated location of said mobile device and said orientation of said mobile device; indicate a selection of said controllable device via said at least one user interface; determine one or more commands associated with said selected controllable device based, at least in part, on user input obtained via said one or more inertial sensors; and initiate transmission of said commands to said selected controllable device via said network interface.

### Claim 37 (depends on 36)

The mobile device as recited in claim 36, and further comprising:
at least one of: a compass, and/or a magnetometer; and
wherein said at least one processing unit to further determine said orientation based, at least in part, on signals associated with at least one of: said one or more inertial sensors, said location service receiver, said compass, and/or said magnetometer.

### Claim 38 (depends on 37)

The mobile device as recited in claim 37, wherein said one or more inertial sensors comprises at least one at least one of: an accelerometer, a gyroscope, and/or a barometer.

### Claim 39 (depends on 1)

The mobile device as recited in claim 1, and wherein said at least one processing unit to further determine said direction based, at least in part, on said estimated location of said mobile device and said orientation.

### Claim 40 (depends on 36)

The mobile device as recited in claim 36, wherein said direction comprises a directional region of space and wherein said at least one processing unit to further determine that said location of said controllable device is within said directional region of space.

### Claim 41 (depends on 36)

The mobile device as recited in claim 36, and wherein said at least one processing unit to further: display information identifying at least said controllable device via said at least one user interface; and obtain user input associated with said selection of said controllable device via at least one of: an interactive or touch-screen display, a microphone, a camera, a button or keypad, and/or at least one of said one or more inertial sensors.

### Claim 42 (depends on 41)

The mobile device as recited in claim 41, and wherein said at least one processing unit to further display information identifying at least one other selectable controllable device via said at least one user interface.

### Claim 43 (depends on 41)

The mobile device as recited in claim 41, and wherein said at least one processing unit to further authenticate that a user may select and/or control said controllable device based, at least in part, on said user input.

### Claim 44 (depends on 36)

The mobile device as recited in claim 36, and wherein said at least one processing unit to further indicate said selection of said controllable device via said at least one user interface, said at least one user interface comprising at least one of: a visual interface; an audio interface; and/or a tactile interface.

### Claim 45 (depends on 36)

The mobile device as recited in claim 36, wherein said one or more commands identify at least one of: at least one physical act performed by a person, and/or at least one physical gesture performed using said mobile device, as determined based, at least in part, on one or more signals associated with said one or more inertial sensors.

### Claim 46 (depends on 45)

The mobile device as recited in claim 45, wherein said at least one physical act results in information representing at least one of an audio waveform and/or an visual image that is identifiable by said at least one processing unit.

### Claim 47 (depends on 46)

The mobile device as recited in claim 46, and further comprising:
at least one of a microphone and/or a camera; and
wherein said at least one physical act is identifiable based, at least in part, on signals associated with at least one of: said microphone and/or said camera.

### Claim 48 (depends on 36)

The mobile device as recited in claim 36, and wherein said at least one processing unit to further obtain said one or more commands associated with said selected controllable device from one or more other devices via said network interface.

### Claim 49 (depends on 36)

The mobile device as recited in claim 36, and wherein said at least one processing unit to further: obtain status information associated with said selected controllable device via said network interface; and present feedback to a user based, at least in part, on said status information.

### Claim 50

A mobile device comprising:
one or more inertial sensors; and
at least one processing unit to: obtain an estimated location of a controllable device; process signals from said one or more inertial sensors of said mobile device to characterize one or more gestures; identify one or more commands for use by said controllable device based, at least in part, on said characterized one or more gestures; and identify said controllable device for receiving said one or more commands responsive to an orientation of said mobile device with a direction toward said estimated location of said controllable device, said direction being determined based, at least in part, on an estimated location of said mobile device and said orientation of said mobile device, wherein said estimated location of said mobile device is based, at least in part, on one or more received signals.

### Claim 51 (depends on 50)

The mobile device as recited in claim 50, and further comprising:
a network interface; and
wherein said at least one processing unit to further obtain signals via said network interface which characterize at least one additional gesture of said one or more gestures.

### Claim 52 (depends on 50)

The mobile device as recited in claim 50, and further comprising:
a network interface; and
wherein said at least one processing unit to further obtain via said network interface at least one of: operational information associated with said controllable device; access control information associated with controllable device; identification information associated with said controllable device; and/or said estimated location of said controllable device.

### Claim 53 (depends on 50)

The mobile device as recited in claim 50, and further comprising:
a camera; and
wherein said at least one processing unit to further obtain via said camera at least one of: image information associated with said controllable device; and/or identification information associated with said controllable device.

### Claim 54 (depends on 50)

The mobile device as recited in claim 50, and further comprising:
a location service receiver; and
wherein said at least one processing unit to further determine said estimated location of said controllable device based, at least in part, on one or more signals associated with said location service receiver.

### Claim 55

An article comprising:
a computer readable medium having computer implementable instructions stored thereon which are executable by at least one processing unit in a mobile device to:
estimate a location of said mobile device based, at least in part, on one or more received signals;
associate an orientation of said mobile device with a direction toward a location of a controllable device, said direction being determined based, at least in part, on said estimated location of said mobile device and said orientation of said mobile device;
indicate a selection of said controllable device;
determine one or more commands associated with said selected controllable device based, at least in part, on user input obtained via said one or more inertial sensors; and
initiate transmission of said commands to said selected controllable device.

### Claim 56 (depends on 55)

The article as recited in claim 55, and wherein said computer implementable instructions are executable by said at least one processing unit to further:
determine said orientation based, at least in part, on signals associated with at least one of: said one or more inertial sensors, a location service receiver, a compass, and/or a magnetometer.

### Claim 57 (depends on 56)

The article as recited in claim 56, wherein said one or more inertial sensors comprises at least one at least one of: an accelerometer, a gyroscope, and/or a barometer.

### Claim 58 (depends on 55)

The article as recited in claim 55, wherein said one or more signals are received with a location service receiver.

### Claim 59 (depends on 55)

The article as recited in claim 55, wherein said direction comprises a directional region of space and wherein said computer implementable instructions are executable by said at least one processing unit to further:
determine that said location of said controllable device is within said directional region of space.

### Claim 60 (depends on 55)

The article as recited in claim 55, and wherein said computer implementable instructions are executable by said at least one processing unit to further:
initiate display of information identifying at least said controllable device; and
obtain user input associated with said selection of said controllable device via at least one of: an interactive or touch-screen display, a microphone, a camera, a button or keypad, and/or at least one of said one or more inertial sensors.

### Claim 61 (depends on 60)

The article as recited in claim 60, and wherein said computer implementable instructions are executable by said at least one processing unit to further:
initiate display of information identifying at least one other selectable controllable device.

### Claim 62 (depends on 60)

The article as recited in claim 60, and wherein said computer implementable instructions are executable by said at least one processing unit to further:
authenticate that a user may select and/or control said controllable device based, at least in part, on said user input.

### Claim 63 (depends on 55)

The article as recited in claim 55, and wherein said computer implementable instructions are executable by said at least one processing unit to further:
initiate an indication said selection of said controllable device via at least one of a visual interface, an audio interface, and/or a tactile interface.

### Claim 64 (depends on 55)

The article as recited in claim 55, wherein said one or more commands identify at least one of: at least one physical act performed by a person, and/or at least one physical gesture performed using said mobile device, as determined based, at least in part, on one or more signals associated with said one or more inertial sensors of said mobile device.

### Claim 65 (depends on 64)

The article as recited in claim 64, wherein said at least one physical act results in at least one of an audio waveform and/or an visual image identifiable by said at least one processing unit executing said computer implementable instructions.

### Claim 66 (depends on 65)

The article as recited in claim 65, wherein said at least one physical act is identified based, at least in part, on signals associated with at least one of: a microphone and/or a camera of said mobile device.

### Claim 67 (depends on 55)

The article as recited in claim 55, and wherein said computer implementable instructions are executable by said at least one processing unit to further:
obtain said one or more commands associated with said selected controllable device from at least one of: previously stored data, said selected controllable device, and/or from one or more other devices.

### Claim 68 (depends on 55)

The article as recited in claim 55, and wherein said computer implementable instructions are executable by said at least one processing unit to further:
obtain status information associated with said selected controllable device via a network interface; and
initiate presentation of feedback to a user based, at least in part, on said status information.

### Claim 69 (depends on 55)

The article as recited in claim 55, and wherein said computer implementable instructions are executable by said at least one processing unit to further:
initiate transmission of one or more signals representing said commands to at least one other device via a network interface, and/or to said selected controllable device via a network interface and/or a direct communication link.

### Claim 70

An article comprising:
a computer readable medium having computer implementable instructions stored thereon which are executable by at least one processing unit in a mobile device to:
obtain an estimated location of a controllable device;
process signals from said one or more inertial sensors of said mobile device to characterize one or more gestures;
identify one or more commands for use by said controllable device based, at least in part, on said characterized one or more gestures; and
identify said controllable device for receiving said one or more commands responsive to an orientation of said mobile device with a direction toward said estimated location of said controllable device, said direction being determined based, at least in part, on an estimated location of said mobile device and said orientation of said mobile device, wherein said estimated location of said mobile device is based, at least in part, on one or more received signals.

### Claim 71 (depends on 70)

The article as recited in claim 70, and wherein said computer implementable instructions are executable by said at least one processing unit to further:
obtain signals which characterize at least one additional gesture of said one or more gestures.

### Claim 72 (depends on 70)

The article as recited in claim 70, and wherein said computer implementable instructions are executable by said at least one processing unit to further:
obtain at least one of: operational information associated with said controllable device; access control information associated with controllable device; identification information associated with said controllable device; and/or said estimated location of said controllable device.

### Claim 73 (depends on 70)

The article as recited in claim 70, and wherein said computer implementable instructions are executable by said at least one processing unit to further:
obtain via a camera of said mobile device at least one of: image information associated with said controllable device; and/or identification information associated with said controllable device.

### Claim 74 (depends on 70)

The article as recited in claim 70, and wherein said computer implementable instructions are executable by said at least one processing unit to further:
determine said estimated location of said controllable device based, at least in part, on one or more signals associated with a location service receiver.