# US 2016/0252968 A1 · Interface elements for managing gesture control

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2016/0252968 A1
- Document ID: US20160252968A1
- Status: Published application
- Kind: A1
- Application number: 15/049,609
- Filed: 2016-02-22
- Published: 2016-09-01
- Earliest claimed: 2012-10-10
- Assignees: Amazon Technologies, Inc.
- Applicants: Amazon Technologies, Inc.
- Inventors: Isaac S. Noble
- Web page: https://patentgrep.com/patent/US20160252968A1

## CPC

- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/005: {Input arrangements through a video camera}
- G06F 3/0304: {Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042)}
- G06F 3/0421: {by interrupting or reflecting a light beam, e.g. optical touch-screen}
- G06F 3/046: by electromagnetic means
- G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence)
- G06F 3/04845: for image manipulation, e.g. dragging, rotation, expansion or change of colour
- G06F 3/0485: Scrolling or panning
- G06F 3/0488: using a touch-screen or digitiser, e.g. input of commands through traced gestures
- G06F 3/04883: for inputting data by handwriting, e.g. gesture or text
- G06V 40/20: Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16)
- H04N 23/90: Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 2203/04108: Touchless 2D- digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface without distance measurement in the Z direction
- G06F 2203/04806: Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Approaches are described for enabling a user to utilize fingers or other features of both hands to interact with an interface on an electronic device. A user can hold an device such as a mobile phone in a first hand, and can use a finger of the other hand to navigate through an interface or other display, such as to move relative to the device to cause the interface to scroll, zoom, or otherwise adjust. The user can use a thumb or finger of the hand holding the device to select or interact with one or more user-selectable elements, objects, or portions of the interface displayed on a display element of the device, such as by touching the screen over one of the user-selectable elements for at least a determined period of time to activate or change a function associated with a gesture used to interact with the interface.

## Claims

### Claim 1

(canceled)

### Claim 2

A computer implemented method, under control of one or more computer systems configured with executable instructions, the method comprising:
determining a status of a selectable element;
identifying an action type based at least in part on the status;
capturing image information using a first sensor;
identifying a gesture input based at least in part on analyzing the image information;
determining an interpretation of the gesture input based at least in part on the action type; and
causing presentation of a graphical user interface based at least in part on the interpretation.

### Claim 3 (depends on 2)

The computer implemented method of claim 2, further comprising detecting movement of the at least one object using at least the image information.

### Claim 4 (depends on 2)

The computer implemented method of claim 2, further comprising causing the presentation of the graphical user interface by at least one of updating, scrolling, panning, or zooming.

### Claim 5 (depends on 2)

The computer implemented method of claim 2, further comprising capturing the image information using one or more cameras.

### Claim 6 (depends on 2)

The computer implemented method of claim 2, further comprising:
capturing additional information using a second sensor; and
identifying the gesture input based at least in part on the image information and the additional information.

### Claim 7 (depends on 6)

The computer implemented method of claim 6, further comprising capturing the additional information using one or more electromagnetic field sensors, charge-coupled devices, motion detection sensors, or infrared sensors.

### Claim 8 (depends on 2)

The computer implemented method of claim 2, further comprising identifying at least one object based at least on one of pattern recognition, image recognition, or object recognition.

### Claim 9

A computing system, comprising:
a first sensor;
at least one processor; and
memory including instructions that, when executed by the at least one processor, cause the computing system to:
determine a status of a selectable element;
identify an action type based at least in part on the status;
capture image information using the first sensor;
identify a gesture input based at least in part on analyzing the image information;
determine an interpretation of the gesture input based at least in part on the action type; and
cause presentation of a graphical user interface based at least in part on the interpretation.

### Claim 10 (depends on 9)

The computing system of claim 9, the selectable element being one of a physical button, a graphical element, or combinations thereof.

### Claim 11 (depends on 9)

The computing system of claim 9, the action type being one of updating, scrolling, panning, or zooming.

### Claim 12 (depends on 9)

The computing system of claim 9, the first sensor comprising one or more cameras.

### Claim 13 (depends on 9)

The computing system of claim 9, further comprising a second sensor, the memory including instructions that, when executed by the at least one processor, further cause the computing system to: use the second sensor to capture additional information;
and identify the gesture input based at least in part on the image information and the additional information.

### Claim 14 (depends on 13)

The computing system of claim 13, the second sensor being one or more electromagnetic field sensors, charge-coupled devices, motion detection sensors, or infrared sensors.

### Claim 15 (depends on 9)

The computing system of claim 9, the memory including instructions that, when executed by the at least one processor, further cause the computing system to: identify at least one object based at least on one of pattern recognition, image recognition, or object recognition.

### Claim 16

A computer implemented method, under the control of one or more computer systems configured with executable instructions, the method comprising:
determining a status of a selectable element;
identifying an action type based at least in part on the status;
capturing image information using a first sensor;
capturing addition information using a second sensor;
identifying a gesture input based at least in part on analyzing the image information and the additional information;
determining an interpretation of the gesture input based at least in part on the action type; and
causing presentation of a graphical user interface based at least in part on the interpretation.

### Claim 17 (depends on 16)

The computer implemented method of claim 16, further comprising detecting movement of the at least one object using at least one of the image information or the additional information.

### Claim 18 (depends on 16)

The computer implemented method of claim 16, further comprising causing the presentation of the graphical user interface by at least one of updating, scrolling, panning, or zooming.

### Claim 19 (depends on 16)

The computer implemented method of claim 16, further comprising capturing the image information using one or more cameras.

### Claim 20 (depends on 16)

The computer implemented method of claim 16, further comprising identifying at least one object based at least on one of pattern recognition, image recognition, or object recognition.

### Claim 21 (depends on 16)

The computer implemented method of claim 16, further comprising capturing the additional information using one or more electromagnetic field sensors, charge-coupled devices, motion detection sensors, or infrared sensors.
