# US 2017/0364198 A1 · Remote hover touch system and method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2017/0364198 A1
- Document ID: US20170364198A1
- Status: Published application
- Kind: A1
- Application number: 15/581,412
- Filed: 2017-04-28
- Published: 2017-12-21
- Earliest claimed: 2016-06-21
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung Electronics Co., Ltd.
- Inventors: Arun Rakesh YOGANANDAN; Stacie HIBINO; Chang Long ZHU JIN
- Web page: https://patentgrep.com/patent/US20170364198A1

## CPC

- G06F 3/0482: Interaction with lists of selectable items, e.g. menus
- G02B 27/017: {Head mounted}
- G06F 1/1626: {with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]}
- G06F 1/163: {Wearable computers, e.g. on a belt}
- G06F 1/1694: {the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/0236: {using selection techniques to select from displayed items}
- G06F 3/0346: with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
- G06F 3/042: by opto-electronic means
- G06F 3/0428: {by sensing at the edges of the touch surface the interruption of optical paths, e.g. an illumination plane, parallel to the touch surface which may be virtual (sensing beam interruptions in a planar beam grid of an optical touch-screen G06F3/0421)}
- G06F 3/044: by capacitive means
- G06F 3/04815: Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object
- G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence)
- G06F 3/04845: for image manipulation, e.g. dragging, rotation, expansion or change of colour
- G06F 3/0488: using a touch-screen or digitiser, e.g. input of commands through traced gestures
- G06F 2203/04101: 2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
- G06F 2203/04106: Multi-sensing digitiser, i.e. digitiser using at least two different sensing technologies simultaneously or alternatively, e.g. for detecting pen and finger, for saving power or for improving position detection
- G06F 2203/04108: Touchless 2D- digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface without distance measurement in the Z direction
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS

## Abstract

A controller includes a touch sensor and a proximity sensor. Information on when and where a user touches the surface of the touch sensor is detected. Additionally, information on pre-touch events is detecting. The pre-touch events correspond to a user's finger hovering over the touch sensor within some maximum depth. A display device generates visualizations based on the hover events and the touch events. This can include providing visual feedback in a hover mode where a user's finger is, relative to the touch sensor. Additionally, various forms of navigation information may be provided in the hover mode, as well as other control information.

## Claims

### Claim 1

An apparatus, comprising:
a touch sensor having a touch surface;
a proximity sensor detecting three dimensional position information within a maximum depth from the touch surface;
a controller configured to monitor the touch sensor and the proximity sensor and communicate three dimensional spatial information to a display device describing a position of an interaction device with respect to the touch surface.

### Claim 2 (depends on 1)

The apparatus of claim 1, wherein the display device is a head mounted display and the apparatus is attached or integrated with the head mounted display.

### Claim 3 (depends on 1)

The apparatus of claim 1, in combination with a six degree of freedom tracking unit, wherein tracking information on position and orientation of the controller is provided with the three dimensional spatial information to the display device.

### Claim 4 (depends on 3)

The apparatus of claim 3, wherein the display device includes a user interface that generates a first visualization for touch events in which the interaction device touches the touch surface and a second visualization for pre-touch events when the interaction device hovers away from the touch surface within the maximum depth.

### Claim 5 (depends on 4)

The apparatus of claim 4, wherein the second visualization indicates where the interaction device is currently hovering as an indicator of when and where the interaction device will touch the touch surface.

### Claim 6

A method, comprising:
receiving three dimensional position information describing a position of a user's finger with respect to a surface of a touch sensor within a maximum depth range from the surface;
generating a user interface based at least in part on the three dimensional position information, including:
generating a first visual feedback when a user touches the surface of the touch sensor; and
generating a second visual feedback for a pre-touch event corresponding to the user's finger being in a region spaced away from the surface of the touch sensor but within the maximum depth range.

### Claim 7 (depends on 6)

The method of claim 6, wherein generating the second visual feedback for the pre-touch event comprises generating navigation information.

### Claim 8 (depends on 6)

The method of claim 6, wherein the first visual feedback and the second visual feedback provides visual feedback of where a user's finger is currently hovering as an indicator of when and where the user's finger will touch the surface of the touch sensor.

### Claim 9

A method, comprising:
receiving three dimensional position information describing, a position of a user's finger with respect to a surface of a touch sensor, wherein the position is determined relative to a plane of the surface and within a maximum threshold depth from the surface;
generating a user interface based at least in part on the three dimensional position information;
in response to the user's finger touching the surface of the touch sensor, determining that a touch down has occurred and generating a first set of user interface interactions; and
in response to the user's finger being spaced away from the surface of the touch sensor but within the maximum threshold depth, determining that the user's finger is hovering above the surface of the touch sensor corresponding to a hover mode and generating a second set of user interface interactions.

### Claim 10 (depends on 9)

The method of claim 9, wherein the user interface comprises a cursor mode in which a cursor has a first representation for the touch down and a second representation for the hover mode.

### Claim 11 (depends on 9)

The method of claim 9, wherein the user interface scales at least one feature of the user interface in the hover mode.

### Claim 12 (depends on 9)

The method of claim 9, wherein the user interface provides visual feedback indicating where a user's finger is currently hovering over the touch sensor as an indicator of when and where the user's finger will touch the surface of the touch sensor.

### Claim 13 (depends on 9)

The method of claim 9, wherein the user interface snaps to the nearest valid widget in the hover mode.

### Claim 14 (depends on 9)

The method of claim 9, wherein the user interface reveals a hidden menu in the hover mode.

### Claim 15 (depends on 9)

The method of claim 9, wherein children user interfaces are generated in the hover mode.

### Claim 16 (depends on 9)

The method of claim 9, wherein a secondary cursor is spawned in the hover mode.

### Claim 17 (depends on 9)

The method of claim 9, wherein the user interface is displayed in a heads up display.

### Claim 18 (depends on 9)

The method of claim 9, further comprising receiving 6 degree of freedom tracking information from a device supporting the touch sensor, wherein a virtual mobile device is displayed having an orientation based on the 6 degree of tracking information and the 3 dimensional position information of the touch sensor is used to enter inputs into the virtual mobile device.

### Claim 19 (depends on 18)

The method of claim 18, wherein alphanumeric information is entered into the virtual mobile phone.

### Claim 20 (depends on 9)

The method of claim 9, further comprising receiving 6 degree of freedom tracking information from a device supporting the touch sensor, wherein a coarse selection of a portion of the user interface is based on the 6 degree of tracking information and the 3 dimensional position information of the touch sensor is used to make a fine grained selection.
