# US 2011/0234503 A1 · Multi-Touch Marking Menus and Directional Chording Gestures

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2011/0234503 A1
- Document ID: US20110234503A1
- Status: Published application
- Kind: A1
- Application number: 12/732,978
- Filed: 2010-03-26
- Published: 2011-09-29
- Earliest claimed: 2010-03-26
- Assignees: Autodesk, Inc.
- Inventors: George FITZMAURICE; Tovi Grossman; Gerard Julian Lepinski
- Web page: https://patentgrep.com/patent/US20110234503A1

## CPC

- G06F 3/042: by opto-electronic means
- G06F 3/0235: {using chord techniques (G06F3/0234 takes precedence)}
- G06F 3/0425: {using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected (tracking a projected light spot to determine a position on a display surface G06F3/0386)}
- G06F 3/04883: for inputting data by handwriting, e.g. gesture or text
- G06F 2203/04808: Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A method for interacting with a multi-touch surface to perform an operation. The method includes detecting that at least one finger is in contact with the multi-touch surface, identifying a chord defined by a combination of fingers associated with the at least one finger in contact with the multi-touch surface, and performing a first function associated with the operation based on the chord. The method may also include integrating the chords into directional movement gestures and using the gestures to control system functionality.

## Claims

### Claim 1

A method for interacting with a multi-touch surface to perform an operation, the method comprising:
detecting that at least one finger is in contact with the multi-touch surface;
identifying a chord defined by a combination of fingers associated with the at least one finger in contact with the multi-touch surface; and
performing a first function associated with the operation based on the chord.

### Claim 2 (depends on 1)

The method of claim 1, wherein the step of performing the first function comprises displaying one or more user interface elements associated with the chord on the multi-touch surface.

### Claim 3 (depends on 2)

The method of claim 2, wherein the one or more user interface elements comprise a radial marking menu.

### Claim 4 (depends on 3)

The method of claim 3, wherein the radial marking menu is offset by a fixed distance from a location at which the at least one finger is in contact with the multi-touch surface.

### Claim 5 (depends on 1)

The method of claim 1, wherein the step of identifying the chord comprises:
determining that five fingers are in contact with the multi-touch surface;
identifying, for each of the five fingers, a contact point on the multi-touch surface associated with the finger;
detecting that zero or more of the five fingers have been lifted away from the multi-touch surface; and
identifying the chord associated with the at least one finger based on associating each finger remaining in contact with the multi-touch surface with a corresponding contact point on the multi-touch surface.

### Claim 6 (depends on 1)

The method of claim 1, wherein the step of identifying the chord comprises:
capturing one or more images of the at least one finger that is in contact with the multi-touch surface;
determining a bounding box within the one or more images that encloses the at least one finger that is in contact with the multi-touch surface; and
analyzing the bounding box to identify the chord associated with the at least one finger that is in contact with the multi-touch surface.

### Claim 7 (depends on 6)

The method of claim 6, wherein the one or more images are captured from beneath the multi-touch surface.

### Claim 8 (depends on 1)

The method of claim 1, further comprising the steps of:
detecting a chording gesture caused by the at least one finger sliding along the multi-touch surface; and
performing a second function associated with the operation based on the chording gesture.

### Claim 9 (depends on 8)

The method of claim 8, wherein the step of performing the first function comprises displaying one or more user interface elements associated with the chord on the multi-touch surface, and wherein the step of performing the second function comprises selecting a user interface element associated with the chording gesture.

### Claim 10

A computer-readable storage medium storing instructions that, when executed by a processor, cause a computing device to interact with a multi-touch surface to perform an operation, by performing the steps of:
detecting that at least one finger is in contact with the multi-touch surface;
identifying a chord defined by a combination of fingers associated with the at least one finger in contact with the multi-touch surface; and
performing a first function associated with the operation based on the chord.

### Claim 11 (depends on 10)

The computer-readable storage medium of claim 10, wherein the step of performing the first function comprises displaying one or more user interface elements associated with the chord on the multi-touch surface.

### Claim 12 (depends on 11)

The computer-readable storage medium of claim 11, wherein the one or more user interface elements comprise a radial marking menu.

### Claim 13 (depends on 12)

The computer-readable storage medium of claim 12, wherein the radial marking menu is offset by a fixed distance from a location at which the at least one finger is in contact with the multi-touch surface.

### Claim 14 (depends on 10)

The computer-readable storage medium of claim 10, wherein the step of identifying the chord comprises:
determining that five fingers are in contact with the multi-touch surface;
identifying, for each of the five fingers, a contact point on the multi-touch surface associated with the finger;
detecting that zero or more of the five fingers have been lifted away from the multi-touch surface; and
identifying the chord associated with the at least one finger based on associating each finger remaining in contact with the multi-touch surface with a corresponding contact point on the multi-touch surface.

### Claim 15 (depends on 10)

The computer-readable storage medium of claim 10, wherein the step of identifying the chord comprises:
capturing one or more images of the at least one finger that is in contact with the multi-touch surface;
determining a bounding box within the one or more images that encloses the at least one finger that is in contact with the multi-touch surface; and
analyzing the bounding box to identify the chord associated with the at least one finger that is in contact with the multi-touch surface.

### Claim 16 (depends on 15)

The computer-readable storage medium of claim 15, wherein the one or more images are captured from beneath the multi-touch surface.

### Claim 17 (depends on 10)

The computer-readable storage medium of claim 10, further comprising the steps of:
detecting a chording gesture caused by the at least one finger sliding along the multi-touch surface; and
performing a second function associated with the operation based on chording gesture.

### Claim 18 (depends on 17)

The computer-readable storage medium of claim 17, wherein the step of performing the first function comprises displaying one or more user interface elements associated with the chord on the multi-touch surface, and wherein the step of performing the second function comprises selecting a user interface element associated with the chording gesture.

### Claim 19

A computer system, comprising:
a processor; and
a memory storing instructions that when executed by the processor cause the computer system to interact with a multi-touch surface to perform an operation, by performing the steps of:
detecting that at least one finger is in contact with the multi-touch surface,
identifying a chord defined by a combination of fingers associated with the at least one finger in contact with the multi-touch surface, and
performing a first function associated with the operation based on the chord.

### Claim 20 (depends on 19)

The computer system of claim 19, wherein performing the first function comprises displaying one or more user interface elements associated with the chord on the multi-touch surface.
