# US 2016/0253068 A1 · Method and apparatus for user interface of input devices

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2016/0253068 A1
- Document ID: US20160253068A1
- Status: Published application
- Kind: A1
- Application number: 15/149,538
- Filed: 2016-05-09
- Published: 2016-09-01
- Earliest claimed: 2007-10-31
- Assignees: Genedics LLC
- Applicants: Genedics LLC
- Inventors: Gene S. Fein; Jackson T. Fein; Edward Merritt
- Web page: https://patentgrep.com/patent/US20160253068A1

## CPC

- G06F 3/04815: Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object
- G02B 27/0093: {with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking}
- G03H 1/0005: {Adaptation of holography to specific applications (holographic optical element G02B5/32; holographic scanner G02B26/106; recognition using holographic mask G06V10/88; holographic memories G11B7/0065, G11C13/042)}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/0304: {Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042)}
- G06F 3/0325: {using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image}
- G06F 3/033: Pointing devices displaced or positioned by the user {, e.g. mice, trackballs, pens or joysticks}; Accessories therefor (digitisers characterised by the transducing means G06F3/041)
- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G09G 3/003: {to produce spatial visual effects}
- G03H 1/2294: {Addressing the hologram to an active spatial light modulator}
- G03H 2001/0061: {in haptic applications when the observer interacts with the holobject}
- G09G 2370/02: Networking aspects
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G03H: HOLOGRAPHIC PROCESSES OR APPARATUS (holograms, e.g. point holograms, used as ordinary optical elements G02B5/32; producing stereoscopic or other three-dimensional effects G02B30/00; diffraction-grating systems G02B27/44; systems using moiré fringes G02B27/60; optical logic elements G02F3/00; stereo-photography G03B35/00; photosensitive materials or processes for photographic purposes G03C; {stereo-photographic or similar processes G03C9/00}; apparatus for processing exposed photographic materials G03D; analogue computers performing mathematical operations with the aid of optical elements G06E3/00; authentication by radiation, of concealed information carried by holograms or diffraction gratings G06K19/16; holographic storage G11B7/0065, G11C13/04; {stereoscopic or other three dimensional effects in television systems H04N13/00})
- G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00)

## Abstract

A 3 dimensional (3-D) user interface system employs: one or more 3-D projectors configured to display an image at a first location in 3-D space; one or more sensors configured to sense user interaction with the image and to provide user interaction information; and a processor configured (i) to receive the user interaction information from the one or more sensors; (ii) to correlate the user interaction with the image; and (iii) to provide one or more indications responsive to a correlation of the user interaction with the image, including displaying the image at a second location in 3-D space.

## Claims

### Claim 1

A system for a 3 dimensional (3-D) user interface, the system comprising:
one or more 3-D projectors configured to display an image at a first location in a 3-D coordinate system;
one or more sensors configured based on quadrilateral angle navigation to sense user interaction with the displayed at least one of 3D or holographic image and to provide user interaction information; and
a digital processor configured:
(i) to receive the user interaction information from the one or more sensors,
(ii) to correlate the user interaction with the image, and
(iii) to provide one or more indications responsive to a correlation of the user interaction with the image, wherein the one or more indications comprise displaying the image at a second location in the 3-D coordinate system, wherein the displayed image is displayed from one or more points in the 3-D coordinate system that are at least one of configured and preconfigured to correspond to at least one point sensed by the one or more sensors.

### Claim 2 (depends on 1)

The system of claim 1, further comprising a communications port coupled to the processor and configured to provide communications interface with a computer system.

### Claim 3 (depends on 1)

The system of claim 1, where the computer readable program codes can further cause the computer to provide at least one of the provided indications to a device on a network.

### Claim 4 (depends on 1)

The system of claim 1, wherein the projectors are configured to display the image as at least one of a 3-D display, in a static state, or a holographic user interface which transforms the computing environment to enable a three dimensional holographic style user interface and display system.

### Claim 5 (depends on 1)

The system of claim 1, where the communicating devices may be used for interconnecting or communicating between two or more components connected to an interconnection medium where the computer readable program codes can further cause the computer to provide at least one of the provided indications to a device on a network.

### Claim 6 (depends on 1)

The system of claim 1, wherein the displayed image comprises a 3-D image in a static state where the 3-D image is displayed in the first location disappears when the 3-D image is moved by the processor to the second position.

### Claim 7 (depends on 1)

The system of claim 1, wherein the displayed image comprises a 3D image in a static state where the 3D image is displayed in the first position forms a trailing continuous motion effect when the 3D image is moved via the processor to the second position.

### Claim 8 (depends on 1)

The system of claim 1, wherein the displayed image comprises a 3-D display of a series of images where the duration of time that each image in the series is displayed is regulated by the system.

### Claim 9 (depends on 1)

The system of claim 1, wherein the displayed image comprises one or more 3-D images displayed simultaneously.

### Claim 10 (depends on 1)

The system of claim 1, wherein the displayed image comprises multiple 3-D images controlled by voice commands where at least one of the multiple 3-D images is controlled by voice commands.

### Claim 11 (depends on 1)

The system of claim 1, wherein the displayed image comprises at least one of one or more 3-D images controlled by voice commands where at least one of the 3-D images is controlled by system 3-D devices or the user interaction can comprise movement of the whole body or one or more body parts of the user from the first location to the second location, thereby creating an effect of the image moving from the first location to the second location responsive to the movement of the whole body or the one or more body parts of the user from the first location to the second location.

### Claim 12 (depends on 1)

The system of claim 1, wherein the displayed image comprises 3-D images controlled by voice commands where at least one of multiple 3-D images is controlled by voice commands in tandem with system 3-D devices.

### Claim 13 (depends on 1)

The system of claim 1, wherein the displayed image comprises 3-D images displayed from one or more software programs.

### Claim 14 (depends on 1)

The system of claim 1, wherein the one or more sensors include at least one of: a location sensor, a motion sensor, and a light sensor.

### Claim 15

A system for a 3 dimensional (3-D) user interface, the system comprising:
one or more 3-D projectors configured to display an image at a first location in a 3-D coordinate system; and
a plurality of sensors configured based on quadrilateral angle navigation to sense user interaction with the displayed image, two or more of the plurality of sensors sensing the user interaction including continuous movements, the two or more of the plurality of sensors configured to scan the 3-D coordinate system, and
a digital processor configured:
(i) to receive the user interaction information from the two or more of the plurality of sensors,
(ii) to correlate the user interaction with the image, and
(iii) to provide one or more indications responsive to a correlation of the user interaction with the image, wherein the one or more indications comprise displaying the image at a second location in the 3-D coordinate system.

### Claim 16 (depends on 15)

The system of claim 15, wherein the plurality of sensors include at least one of: a location sensor, a motion sensor, and a light sensor.

### Claim 17

The system of claim of 15, wherein the plurality of sensors includes one or more laser sensors, the one or more laser sensors being configured to geometrically identify a position within the 3-D coordinate system based upon triangulation or quadrilation.

### Claim 18 (depends on 15)

The system of claim 15, wherein the plurality of sensors includes one or more light sensors, the one or more light sensors including at least one of: a photovoltaic sensor, an image sensor and a photo-electric light sensor.

### Claim 19

A system for a 3 dimensional (3-D) user interface, the system comprising:
one or more 3-D projectors configured to display a moving image at a first location in a 3-D coordinate system; and
one or more sensors configured based on quadrilateral angle navigation to sense user interaction with the displayed moving image and to provide user interaction information; and
a digital processor configured:
(i) to receive the user interaction information from the one or more of sensors,
(ii) to correlate the user interaction with the moving image, and
(iii) to provide one or more indications responsive to a correlation of the user interaction with the moving image, wherein the one or more indications comprise displaying the moving image at a second location in the 3-D coordinate system.

### Claim 20 (depends on 19)

The system of claim 19, wherein the one or more 3-D projectors include two or more 3-D projectors that are configured to change the position of display of the moving image at a first location in the 3-D coordinate system, each one of the two or more 3-D projectors being aligned to a fixed space within the 3-D coordinate system.

### Claim 21 (depends on 19)

The system of claim 19, wherein the user interaction includes at least one of: user interaction with a physical apparatus or user interaction based upon a voice of the user.
