# US 11,029,784 B2 · Methods and apparatuses for applying free space inputs for surface constrained controls

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,029,784 B2
- Document ID: US11029784B2
- Status: Granted
- Kind: B2
- Application number: 16/811,326
- Filed: 2020-03-06
- Published: 2020-09-03
- Granted: 2021-06-08
- Earliest claimed: 2015-05-15
- Assignees: Atheer, Inc.
- Applicants: Atheer, Inc.
- Inventors: Shashwat Kandadai; Nathan Abercrombie; Yu-Hsiang Chen; Sleiman Itani
- Web page: https://patentgrep.com/patent/US11029784B2

## CPC

- G06F 3/0425: {using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected (tracking a projected light spot to determine a position on a display surface G06F3/0386)}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/0346: with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
- G06F 3/0416: {Control or interface arrangements specially adapted for digitisers}
- G06F 3/0488: using a touch-screen or digitiser, e.g. input of commands through traced gestures
- G06F 3/04883: for inputting data by handwriting, e.g. gesture or text
- G06F 3/04815: Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object
- G06T 19/00: Manipulating three-dimensional [3D] models or images for computer graphics
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A method, system, apparatus, and/or device for sensing an input in an augmented reality construct. The method, system, apparatus, and/or device may include a processing device and a sensor. The processing device may be configured to determine a virtual boundary within an augmented reality construct, define a first portion of the virtual boundary within the augmented reality construct, and define a second portion of the virtual boundary within the augmented reality construct. The sensor may be sense a first input from an end-effector within the first portion of the virtual boundary and sense a second input from the end-effector within the second portion of the virtual boundary, where the processing device may further generate a device input that corresponds to the first input and the second input and send the device input to another device.

## Claims

### Claim 1

A method, comprising:
determining, by a processing device, a surface-constrained input standard for a surface-constrained input generated approximate to a physical surface;
determining, by the processing device, is a virtual boundary within an augmented reality construct;
defining, by the processing device, a first portion of the virtual boundary to receive a first input;
defining, by the processing device, a second portion of the virtual boundary to receive a second input;
sensing, by a sensor, a first input within the first portion of the virtual boundary;
sensing, by the sensor, a second input within the second portion of the virtual boundary;
generating, by the processing device, a device input that corresponds to the first input or the second input; and
sending the device input to a device.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
determining, by the processing device, a first input standard for the first input; or
determining, by the processing device, a second input standard for the second input.

### Claim 3 (depends on 2)

The method of claim 2, wherein:
the first input standard defines a touch down motion where an end-effector extends downwardly from a first plane to a second plane; and
the second input standard defines:
a swipe motion where the end-effector extends along the second plane from a first point to a second point; or
a liftoff motion where the end-effector extends upwardly from the second plane to the first plane or a third plane.

### Claim 4 (depends on 3)

The method of claim 3, wherein the end-effector comprises one or more fingers of an individual.

### Claim 5 (depends on 3)

The method of claim 3, wherein:
the touch down motion extends downwardly at a first angle.

### Claim 6 (depends on 3)

The method of claim 3, further comprising:
determining that the first input meets the first input standard; or
determining that the second input meets the second input standard.

### Claim 7 (depends on 6)

The method of claim 6, further comprising generating the device input when the first input meets the first input standard or the second input meets the second input standard.

### Claim 8 (depends on 1)

The method of claim 1, wherein a width of the second portion of the virtual boundary is substantially similar to a width of the physical surface.

### Claim 9 (depends on 1)

The method of claim 1, wherein the device is configured to receive the surface-constrained input.

### Claim 10 (depends on 1)

The method of claim 1, wherein the device is configured to receive an analog touch input that conforms to an analog touch input standard.

### Claim 11 (depends on 1)

The method of claim 1, further comprising identifying the surface-constrained input that is associated with the first input or the second input, wherein the device input corresponds to the surface-constrained input.

### Claim 12 (depends on 1)

The method of claim 1, wherein the surface-constrained input is an input received on the surface of a touchscreen device.

### Claim 13

A device, comprising:
a processing device configured to:
determine a virtual boundary within an augmented reality construct; and
define a portion of the virtual boundary within the augmented reality construct configured to receive and end-effector input within the augmented reality construct; and
a sensor coupled to the processing device, wherein the sensor is configured to:
sense the end-effector input from an end-effector within the portion of the virtual boundary
wherein the processing device is further configured to:
generate a device input that corresponds to the end-effector input; and
send the device input to another device.

### Claim 14 (depends on 13)

The device of claim 13, wherein the processing device is further configured to associate a surface-constrained input with the device input, wherein the surface-constrained input is an input generated approximate to a physical surface.

### Claim 15 (depends on 13)

The device of claim 13, wherein the processing device is further configured to limit a bounce acceleration of the end-effector input, wherein the bounce acceleration is a motion indicative of a downward motion that is rapidly reversed to an upward motion.

### Claim 16 (depends on 13)

The device of claim 13, wherein the end-effector input comprises at least one of:
a touch down motion where the end-effector extends downwardly from a first plane to a second plane;
a liftoff motion where the end-effector extends upwardly from the second plane to the first plane or a third plane; or
a swipe motion where the end-effector extends along the second plane from a first point to a second point.

### Claim 17 (depends on 16)

The device of claim 16, wherein the processing device is further configured to:
determine a threshold distance range for the end-effector to extend along the second plane; and
generate the device input when the end-effector extends along the second plane within the threshold distance range.

### Claim 18 (depends on 16)

The device of claim 16, wherein:
the end-effector is a hand of a user; and
the processing device is further configured to:
determine a threshold distance range for a knuckle of the hand to extend along the second plane; and
generate the device input when the knuckle extends along the second plane within the threshold distance range.

### Claim 19

An apparatus, comprising:
a processing device configured to:
determine a virtual boundary within an augmented reality construct;
define a portion of the virtual boundary within the augmented reality construct configured to receive an end-effector input; and
a sensor coupled to the processing device, wherein the sensor is configured to:
sense an end-effector input from an end-effector within the portion of the virtual boundary, the end-effector input being a motion where an end-effector moves from a first plane to a second plane,
wherein the processing device is further configured to:
generate a device input that corresponds to the end-effector input; and
send the device input to a device.

### Claim 20 (depends on 19)

The apparatus of claim 19, wherein the processing device is further configured to:
determine a threshold distance range for the end-effector to move from the first plane to the second plane; and
generate the device input when the end-effector moves from the first plane to the second plane within the threshold distance range.