# US 2013/0154952 A1 · Gesture combining multi-touch and movement

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2013/0154952 A1
- Document ID: US20130154952A1
- Status: Published application
- Kind: A1
- Application number: 13/327,794
- Filed: 2011-12-16
- Published: 2013-06-20
- Earliest claimed: 2011-12-16
- Assignees: Microsoft Corporation
- Inventors: Kenneth P. Hinckley; Hyunyoung Song
- Web page: https://patentgrep.com/patent/US20130154952A1

## CPC

- G06F 1/1694: {the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer}
- G06F 3/0346: with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
- G06F 3/04166: {Details of scanning methods, e.g. sampling time, grouping of sub areas or time sharing with display driving (Synchronisation with the driving of the display or the backlighting unit to avoid interferences generated internally G06F3/04184)}
- G06F 3/0484: for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F 3/04883: for inputting data by handwriting, e.g. gesture or text
- G06F 2203/04106: Multi-sensing digitiser, i.e. digitiser using at least two different sensing technologies simultaneously or alternatively, e.g. for detecting pen and finger, for saving power or for improving position detection
- G06F 2203/04806: Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
- G06F 2203/04808: Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Functionality is described herein for interpreting gestures made by a user in the course of interacting with a handheld computing device. The functionality operates by: (a) receiving a touch input event from at least one touch input mechanism; (b) receiving a movement input event from at least one movement input mechanism in response to movement of the computing device; and (c) determining whether the touch input event and the movement input event indicate that a user has made a multi-touch-movement (MTM) gesture. A user performs a MTM gesture by touching a surface of the touch input mechanism to establish two or more contacts in conjunction with moving the computing device in a prescribed manner. The functionality can define an action space in response to the MTM gesture and perform an action which affects the action space.

## Claims

### Claim 1

A method, performed by a handheld computing device, for responding to input events, comprising:
receiving a touch input event from at least one touch input mechanism;
receiving a movement input event from at least one movement input mechanism in response to movement of the computing device;
determining whether the touch input event and the movement input event indicate that a user has performed a multi-touch-movement gesture,
where the multi-touch-movement gesture entails establishing two or more contacts with a surface of the touch input mechanism, in conjunction with moving the computing device in a prescribed manner;
defining an action space which is demarcated by said two or more contacts; and
performing an operation that affects the action space.

### Claim 2 (depends on 1)

The method of claim 1, wherein said at least one touch input mechanism comprises a touchscreen interface mechanism having a display surface that is disposed on at least one surface of the computing device.

### Claim 3 (depends on 1)

The method of claim 1, wherein said at least one movement input mechanism comprises at least one of:
an accelerometer device;
a gyroscope device; and
a magnetometer device.

### Claim 4 (depends on 1)

The method of claim 1, wherein said two or more contacts define two opposing corners of the action space.

### Claim 5 (depends on 1)

The method of claim 1, further comprising displaying at least one prompt that guides the user as to placement of a contact on the surface of the touch input mechanism.

### Claim 6 (depends on 1)

The method of claim 1, wherein said determining comprises:
determining that the user has made a first multi-touch-movement gesture if the user contacts first regions of the surface of the touch input mechanism; and
determining that the user has made a second multi-touch-movement gesture if the user contacts second regions of the surface of the touch input mechanism, the first regions differing from the second regions, at least in part,
the first multi-touch-movement gesture invoking a first action and the second multi-touch-movement gesture invoking a second action, the first action being different than the second action.

### Claim 7 (depends on 6)

The method of claim 6, wherein the first regions are associated with a first corner and a second corner of the action space, and the second regions are associated with a third corner and a fourth corner of the action space, wherein the first and second corners differ from the third and fourth corners at least in part.

### Claim 8 (depends on 1)

The method of claim 1, wherein said determining also comprises:
determining a spatial shift of any of said two or more contacts during movement of the computing device; and
determining whether the spatial shift is below a prescribed threshold, and concluding that a user continues to perform the multi-touch-movement gesture if the spatial shift is below the prescribed threshold.

### Claim 9 (depends on 1)

The method of claim 1, wherein said determining comprises:
determining whether movement of the computing device is indicative of handling the computing device by the user for a non-input-related purpose, to provide a handling input event; and
determining that the user has made the multi-touch-movement gesture based, in part, on the handling input event.

### Claim 10 (depends on 1)

The method of claim 1, wherein the prescribed movement corresponds to a tilting movement of the computing device whereby the computing device is rotated about at least one axis from a starting position.

### Claim 11 (depends on 1)

The method of claim 1, wherein the prescribed movement corresponds to a tilting movement of the computing device whereby the computing device is rotated about at least one axis from a starting position and then rotated back to the starting position.

### Claim 12 (depends on 1)

The method of claim 1, wherein the prescribed movement corresponds to at least one of:
a prescribed vibratory movement;
a prescribed lateral displacement movement in a plane;
a prescribed shaking movement; and
a prescribed tapping movement.

### Claim 13 (depends on 1)

The method of claim 1, wherein said determining also comprises:
determining that the user has made a first multi-touch-movement gesture if the user moves the computing device in a first prescribed manner; and
determining that the user has made a second multi-touch-movement gesture if the user moves the computing device is a second prescribed manner,
the first multi-touch-movement gesture invoking a first action and the second multi-touch-movement gesture invoking a second action, the first action being different than the second action.

### Claim 14 (depends on 1)

The method of claim 1, further comprising selecting an object identified by said two or more contacts.

### Claim 15 (depends on 1)

The method of claim 1, further comprising:
prior to detecting that the user has executed the multi-touch-movement gesture, detecting that a user has executed a preliminary gesture which involves contacting the surface of the touch input device with said two or more contacts,
wherein the user executes the multi-touch-movement gesture without removing said two or more contacts established by the preliminary gesture.

### Claim 16 (depends on 15)

The method of claim 15, wherein the preliminary gesture is a zooming, scrolling, or panning gesture.

### Claim 17

A computer readable storage medium for storing computer readable instructions, the computer readable instructions providing an interpretation and behavior selection module (IBSM), implemented by a handheld computing device, when the instructions are executed by one or more processing devices, the computer readable instructions comprising:
logic configured to receive a touch input event from at least one touch input mechanism;
logic configured to receive a movement input event from at least one movement input mechanism in response to movement of the computing device;
logic configured to determine whether the touch input event and the movement input event indicate that a user has made a multi-touch-movement gesture by:
determining that the user has applied at least two contacts on a surface of the touch input mechanism to demarcate an action space on the display surface; and
determining that the user has moved the computing device in a prescribed manner while touching the surface with said at least two contacts; and
logic configured to select an object associated with the action space in response to the multi-touch-movement gesture.

### Claim 18 (depends on 17)

The computer readable storage medium of claim 17, wherein the prescribed movement corresponds to a tilting movement of the computing device whereby the computing device is rotated about at least one axis from a starting position.

### Claim 19

An interpretation and behavior selection module, implemented by computing functionality, for interpreting user interaction with a handheld computing device, comprising:
a gesture mapping module configured to receive:
a touch input event from at least one touch input mechanism; and
a movement input event from at least one movement input mechanism that describes movement of the computing device; and
a data store for storing signatures associated with different indicative ways that a user can interact with the computing device, the signatures comprising at least:
a multi-touch-movement signature that provides information which characterizes a multi-touch-movement gesture that a user makes by touching a surface of the touch input mechanism with at least two contacts while moving the computing device in a prescribed manner; and
a handling movement signature that provides information which characterizes a manner in which the user handles the computing device for a non-input-related purpose,
the gesture matching module further configured to determine whether the user has made a multi-touch-movement gesture by comparing the touch input event and the movement input event against the signatures provided in the data store,
where at least two multi-touch-motion gestures invoke different respective actions depending on at least one of:
a manner in which the user touches the computing device, as reflected by the touch input event; and
a manner in which the user moves the computing device, as reflected by the movement input event.

### Claim 20 (depends on 19)

An interpretation and behavior selection module of claim 19, wherein the prescribed movement associated with the multi-touch-movement signature corresponds to a tilting movement of the computing device whereby the computing device is rotated about at least one axis from a starting position.