# US 11,175,749 B2 · Three-dimensional man/machine interface

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,175,749 B2
- Document ID: US11175749B2
- Status: Granted
- Kind: B2
- Application number: 16/398,321
- Filed: 2019-04-30
- Published: 2019-11-28
- Granted: 2021-11-16
- Earliest claimed: 2011-01-31
- Assignees: Quickstep Technologies LLC
- Applicants: Quickstep Technologies LLC
- Inventors: Didier Roziere
- Web page: https://patentgrep.com/patent/US11175749B2

## CPC

- G06F 3/033: Pointing devices displaced or positioned by the user {, e.g. mice, trackballs, pens or joysticks}; Accessories therefor (digitisers characterised by the transducing means G06F3/041)
- G06F 3/044: by capacitive means
- G06F 3/04815: Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object
- G06F 3/04817: using icons (graphical or visual programming using iconic symbols G06F8/34)
- G06F 3/0482: Interaction with lists of selectable items, e.g. menus
- G06F 3/0486: Drag-and-drop
- G06F 3/0488: using a touch-screen or digitiser, e.g. input of commands through traced gestures
- G06F 3/04883: for inputting data by handwriting, e.g. gesture or text
- G06F 2203/04101: 2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A method is provided for selecting controls, which implement a control interface, a display, and at least one sensor capable of detecting at least one control, object, including a step (i) of obtaining information on the distance between the control object(s) and the control interface using the sensor(s), and a step (ii) of displaying, on the display, at least one symbol representing a control or set of controls according to a display mode, wherein the method further includes a step of using the distance information to determine the display mode of the symbol(s). A device is also provided for implementing the method and to an apparatus.

## Claims

### Claim 1

A method comprising:
at an electronic device with a touch screen:
while displaying a symbol on the touch screen, detecting, via the touch screen, a first hovering input that includes detecting objects including a first object and a second object over the touch screen without detecting the objects in contact with the touch screen;
in response to detecting that the first hovering input meets one or more first criteria at a respective location of the touch screen corresponding to the symbol including a first criterion that is satisfied when the first object and the second object move closer together while over a portion of the touch screen that includes the symbol, selecting the symbol;
while the symbol is selected, detecting, via the touch screen, a second hovering input that includes detecting movement of the objects over the touch screen without detecting the objects in contact with the touch screen;
in response to detecting that the second hovering input meets one or more second criteria, moving the symbol displayed on the touch screen in accordance with the movement of the objects;
after moving the symbol and while the symbol is selected, detecting, via the touch screen, a third hovering input that includes detecting movement of the objects over the touch screen without detecting the objects in contact with the touch screen; and
in response to detecting the third hovering input meets one or more third criteria, including a second criterion that is satisfied when the first object and the second object move further apart while over a portion of the touch screen that includes the symbol, deselecting the symbol displayed on the touch screen.

### Claim 2 (depends on 1)

The method of claim 1, wherein the one or more first criteria include a third criterion that is satisfied when the objects include two or more fingers.

### Claim 3 (depends on 1)

The method of claim 1, wherein the one or more first criteria include a third criterion that is satisfied when the objects are within a threshold distance from the touch screen.

### Claim 4 (depends on 1)

The method of claim 1, wherein the one or more first criteria include a third criterion that is satisfied when the first object and the second object of the first hovering input converge toward the respective location.

### Claim 5 (depends on 1)

The method of claim 1, wherein the one or more first criteria include a third criterion that is satisfied when the objects are detected within a threshold distance from the touch screen for a threshold duration.

### Claim 6 (depends on 1)

The method of claim 1, wherein moving the symbol displayed on the touch screen from the respective location in accordance with the movement of the objects includes moving the symbol with the objects as the objects move over the touch screen.

### Claim 7 (depends on 1)

The method of claim 1, wherein the one or more second criteria include a third criterion that is satisfied when the second hovering input includes a rotation movement of the objects, and wherein moving the symbol includes rotating the symbol at the respective location.

### Claim 8 (depends on 1)

The method of claim 1, wherein the one or more second criteria include a third criterion that is satisfied when a distance between the first object and the second object is less than a threshold distance during the second hovering input, and that is not satisfied when the distance between the first object and the second object is greater than the threshold distance during the second hovering input.

### Claim 9 (depends on 1)

The method of claim 1, further comprising:
in response to detecting that the second hovering input fails to meet the one or more second criteria, forgoing moving the symbol or returning the symbol displayed on the touch screen to the respective location on the touch screen.

### Claim 10 (depends on 1)

The method of claim 1, wherein the one or more third criteria include a third criterion that is satisfied when the third hovering input includes a movement of the first object and the second object apart beyond a threshold distance.

### Claim 11 (depends on 1)

The method of claim 1, further comprising:
while moving the symbol, detecting, via the touch screen, a fourth hovering input that includes detecting movement of the objects over the touch screen without detecting the objects in contact with the touch screen; and
in response to detecting that the fourth hovering input meets one or more fourth criteria, freezing movement of the symbol, changing an appearance of the symbol, ceasing display of the symbol on the touch screen, or returning the symbol to the respective location.

### Claim 12 (depends on 11)

The method of claim 11, wherein the one or more fourth criteria include a third criterion that is satisfied when the objects include two or more objects and a distance between the two or more objects is greater than a threshold distance.

### Claim 13

A non-transitory computer-readable storage medium storing instructions, which when executed by an electronic device with a touch screen, cause the electronic device to perform a method comprising:
while displaying a symbol on the touch screen, detecting, via the touch screen, a first hovering input that includes detecting objects including a first object and a second object over the touch screen without detecting the objects in contact with the touch screen;
in response to detecting that the first hovering input meets one or more first criteria at a respective location of the touch screen corresponding to the symbol including a first criterion that is satisfied when the first object and the second object move closer together while over a portion of the touch screen that includes the symbol, selecting the symbol;
while the symbol is selected, detecting, via the touch screen, a second hovering input that includes detecting movement of the objects over the touch screen without detecting the objects in contact with the touch screen;
in response to detecting that the second hovering input meets one or more second criteria, moving the symbol displayed on the touch screen in accordance with the movement of the objects;
after moving the symbol and while the symbol is selected, detecting, via the touch screen, a third hovering input that includes detecting movement of the objects over the touch screen without detecting the objects in contact with the touch screen; and
in response to detecting the third hovering input meets one or more third criteria, including a second criterion that is satisfied when the first object and the second object move further apart while over a portion of the touch screen that includes the symbol, deselecting the symbol displayed on the touch screen.

### Claim 14 (depends on 13)

The non-transitory computer readable storage medium of claim 13, wherein the one or more first criteria include a third criterion that is satisfied when the objects include two or more fingers.

### Claim 15 (depends on 13)

The non-transitory computer readable storage medium of claim 13, wherein the one or more first criteria include a third criterion that is satisfied when the objects are within a threshold distance from the touch screen.

### Claim 16 (depends on 13)

The non-transitory computer readable storage medium of claim 13, wherein the one or more first criteria include a third criterion that is satisfied when the first object and the second object of the first hovering input converge toward the respective location.

### Claim 17 (depends on 13)

The non-transitory computer readable storage medium of claim 13, wherein the one or more first criteria include a third criterion that is satisfied when the objects are detected within a threshold distance from the touch screen for a threshold duration.

### Claim 18 (depends on 13)

The non-transitory computer readable storage medium of claim 13, wherein moving the symbol displayed on the touch screen from the respective location in accordance with the movement of the objects includes moving the symbol with the objects as the objects move over the touch screen.

### Claim 19 (depends on 13)

The non-transitory computer readable storage medium of claim 13, wherein the one or more second criteria include a third criterion that is satisfied when the second hovering input includes a rotation movement of the objects, and wherein moving the symbol includes rotating the symbol at the respective location.

### Claim 20 (depends on 13)

The non-transitory computer readable storage medium of claim 13, wherein the one or more second criteria include a third criterion that is satisfied when the objects include two objects, and a distance between the first object and the second object is less than a threshold distance during the second hovering input, and that is not satisfied when the distance between the first object and the second object is greater than the threshold distance during the second hovering input.

### Claim 21 (depends on 13)

The non-transitory computer readable storage medium of claim 13, the method further comprising:
in response to detecting that the second hovering input fails to meet the one or more second criteria, forgoing moving the symbol or returning the symbol displayed on the touch screen to the respective location on the touch screen.

### Claim 22 (depends on 13)

The non-transitory computer readable storage medium of claim 13, wherein the one or more third criteria include a third criterion that is satisfied when the third hovering input includes a movement of the first object and the second object apart beyond a threshold distance.

### Claim 23 (depends on 13)

The non-transitory computer readable storage medium of claim 13, the method further comprising:
while moving the symbol, detecting, via the touch screen, a fourth hovering input that includes detecting movement of the objects over the touch screen without detecting the objects in contact with the touch screen; and
in response to detecting that the fourth hovering input meets one or more fourth criteria, freezing movement of the symbol, changing an appearance of the symbol, ceasing display of the symbol on the touch screen, or returning the symbol to the respective location.

### Claim 24 (depends on 23)

The non-transitory computer readable storage medium of claim 23, wherein the one or more fourth criteria include a third criterion that is satisfied when the objects include two or more objects and a distance between the two or more objects is greater than a threshold distance.

### Claim 25

An electronic device comprising:
a touch screen;
one or more processors; and
memory storing instructions, which when executed by the one or more processors, cause the one or more processors to perform a method comprising:
while displaying a symbol on the touch screen, detecting, via the touch screen, a first hovering input that includes detecting objects including a first object and a second object over the touch screen without detecting the objects in contact with the touch screen;
in response to detecting that the first hovering input meets one or more first criteria at a respective location of the touch screen corresponding to the symbol including a first criterion that is satisfied when the first object and the second object move closer together while over a portion of the touch screen that includes the symbol, selecting the symbol;
while the symbol is selected, detecting, via the touch screen, a second hovering input that includes detecting movement of the objects over the touch screen without detecting the objects in contact with the touch screen;
in response to detecting that the second hovering input meets one or more second criteria, moving the symbol displayed on the touch screen in accordance with the movement of the objects;
after moving the symbol and while the symbol is selected, detecting, via the touch screen, a third hovering input that includes detecting movement of the objects over the touch screen without detecting the objects in contact with the touch screen; and
in response to detecting the third hovering input meets one or more third criteria, including a second criterion that is satisfied when the first object and the second object move further apart while over a portion of the touch screen that includes the symbol, deselecting the symbol displayed on the touch screen.

### Claim 26 (depends on 25)

The electronic device of claim 25, wherein the one or more first criteria include a third criterion that is satisfied when the objects include two or more fingers.

### Claim 27 (depends on 25)

The electronic device of claim 25, wherein the one or more first criteria include a third criterion that is satisfied when the objects are within a threshold distance from the touch screen.

### Claim 28 (depends on 25)

The electronic device of claim 25, wherein the one or more first criteria include a third criterion that is satisfied when the first object and the second object of the first hovering input converge toward the respective location.

### Claim 29 (depends on 25)

The electronic device of claim 25, wherein the one or more first criteria include a third criterion that is satisfied when the objects are detected within a threshold distance from the touch screen for a threshold duration.

### Claim 30 (depends on 25)

The electronic device of claim 25, wherein moving the symbol displayed on the touch screen from the respective location in accordance with the movement of the objects includes moving the symbol with the objects as the objects move over the touch screen.

### Claim 31 (depends on 25)

The electronic device of claim 25, wherein the one or more second criteria include a third criterion that is satisfied when the second hovering input includes a rotation movement of the objects, and wherein moving the symbol includes rotating the symbol at the respective location.

### Claim 32 (depends on 25)

The electronic device of claim 25, wherein the one or more second criteria include a third criterion that is satisfied when the objects include two objects, and a distance between the first object and the second object is less than a threshold distance during the second hovering input, and that is not satisfied when the distance between the first object and the second object is greater than the threshold distance during the second hovering input.

### Claim 33 (depends on 25)

The electronic device of claim 25, the method further comprising:
in response to detecting that the second hovering input fails to meet the one or more second criteria, forgoing moving the symbol or returning the symbol displayed on the touch screen to the respective location on the touch screen.

### Claim 34 (depends on 25)

The electronic device of claim 25, wherein the one or more third criteria include a third criterion that is satisfied when the third hovering input includes a movement of the first object and the second object apart beyond a threshold distance.

### Claim 35 (depends on 25)

The electronic device of claim 25, the method further comprising:
while moving the symbol, detecting, via the touch screen, a fourth hovering input that includes detecting movement of the objects over the touch screen without detecting the objects in contact with the touch screen; and
in response to detecting that the fourth hovering input meets one or more fourth criteria, freezing movement of the symbol, changing an appearance of the symbol, ceasing display of the symbol on the touch screen, or returning the symbol to the respective location.

### Claim 36 (depends on 35)

The electronic device of claim 35, wherein the one or more fourth criteria include a third criterion that is satisfied when the objects include two or more objects and a distance between the two or more objects is greater than a threshold distance.