# US 2017/0287225 A1 · Interactions with 3d virtual objects using poses and multiple-dof controllers

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2017/0287225 A1
- Document ID: US20170287225A1
- Status: Published application
- Kind: A1
- Application number: 15/473,444
- Filed: 2017-03-29
- Published: 2017-10-05
- Earliest claimed: 2016-03-31
- Assignees: Magic Leap, Inc.
- Applicants: Magic Leap, Inc.
- Inventors: James M. Powderly; Savannah Niles; Frank Hamilton; Marshal A. Fontaine; Paul Armistead Hoover
- Web page: https://patentgrep.com/patent/US20170287225A1

## CPC

- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/0346: with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
- G06F 3/04815: Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object
- G06F 1/163: {Wearable computers, e.g. on a belt}
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A wearable system can comprise a display system configured to present virtual content in a three-dimensional space, a user input device configured to receive a user input, and one or more sensors configured to detect a user's pose. The wearable system can support various user interactions with objects in the user's environment based on contextual information. As an example, the wearable system can adjust the size of an aperture of a virtual cone during a cone cast (e.g., with the user's poses) based on the contextual information. As another example, the wearable system can adjust the amount of movement of virtual objects associated with an actuation of the user input device based on the contextual information.

## Claims

### Claim 1

A system for interacting with objects for a wearable device, the system comprising:
a display system of a wearable device configured to present a three-dimensional (3D) view to a user and permit a user interaction with objects in a field of regard (FOR) of a user, the FOR comprising a portion of the environment around the user that is capable of being perceived by the user via the display system;
a sensor configured to acquire data associated with a pose of the user;
a hardware processor in communication with the sensor and the display system, the hardware processor programmed to:
determine a pose of the user based on the data acquired by the sensor;
initiate a cone cast on a group of objects in the FOR, the cone cast comprises casting a virtual cone with an aperture in a direction based at least partly on the pose of the user;
analyze contextual information associated with the user's environment;
update the aperture of the virtual cone based at least partly on the contextual information; and
render a visual representation of the virtual cone for the cone cast.

### Claim 2 (depends on 1)

The system of claim 1, wherein the contextual information comprises at least one of: a type, a layout, a location, a size, or a density of a subgroup of objects within the field of view (FOV) of the user, wherein the FOV comprises a portion of the FOR that is capable of being perceived at a given time by the user via the display system.

### Claim 3 (depends on 2)

The system of claim 2, wherein the density of the subgroup of objects within the FOV of the user is calculated by at least one of:
calculating a number of objects in the subgroup of objects;
calculating a percentage of the FOV that is covered by the subgroup of objects; or
calculating a contour map for objects in the subgroup of objects.

### Claim 4 (depends on 1)

The system of claim 1, wherein the hardware processor is further programmed to detect a collision between the virtual cone and one or more objects within the group of objects in the FOR, and wherein in response to detecting the collision, the hardware processor is further programmed to present a focus indicator to the one or more objects.

### Claim 5 (depends on 4)

The system of claim 4, wherein the hardware processor is programmed to apply an occlusion disambiguation technique to the one or more objects collided with the virtual cone to identify an occluded object.

### Claim 6 (depends on 1)

The system of claim 1, wherein the cone comprises a central ray and wherein the aperture is transverse to the central ray.

### Claim 7 (depends on 1)

The system of claim 1, wherein the virtual cone comprises a proximal end and wherein the proximal end is anchored to at least one of the following locations: a location in-between the user's eyes, a location on a portion of a user's arm, a location on a user input device, or any other location in the environment of the user.

### Claim 8 (depends on 1)

The system of claim 1, wherein the hardware processor is further programmed to receive an indication from a user input device anchoring a depth of the virtual cone to a depth plane and wherein cone cast is performed on the group of objects within the depth plane.

### Claim 9

A method for interacting with objects for a wearable device, the method comprising:
receiving a selection of a target virtual object displayed to a user at a first position in a three-dimensional (3D) space;
receiving an indication of a movement for the target virtual object;
analyzing contextual information associated with the target virtual object;
calculating a multiplier to be applied to a movement of the target virtual object based at least partly on the contextual information;
calculating a movement amount for the target virtual object, the movement amount based at least partly on the indication of the movement and the multiplier; and
displaying, to the user, the target virtual object at a second position, the second position based at least in part on the first position and the movement amount.

### Claim 10 (depends on 9)

The method of claim 9, wherein the contextual information comprises a distance from the user to the target virtual object.

### Claim 11 (depends on 10)

The method of claim 10, wherein the multiplier increases proportionally with an increase in the distance.

### Claim 12 (depends on 9)

The method of claim 9, wherein the movement comprises one or more of: a position change, a speed, or an acceleration.

### Claim 13 (depends on 9)

The method of claim 9, wherein the indication of the movement comprises at least one of: an actuation of a user input device associated with the wearable device or a change in a pose of the user.

### Claim 14 (depends on 13)

The method of claim 13, wherein the pose comprises one or more of: a head pose, an eye pose, or a body pose.

### Claim 15

A system for interacting with objects for a wearable device, the system comprising:
a display system of a wearable device configured to present a three-dimensional (3D) view of to a user, the 3D view comprising a target virtual object;
a hardware processor in communication with the display system, the hardware processor programmed to:
receive an indication of a movement for the target virtual object;
analyze contextual information associated with the target virtual object;
calculate a multiplier to be applied to a movement of the target virtual object based at least partly on the contextual information;
calculate a movement amount for the target virtual object, the movement amount based at least partly on the indication of the movement and the multiplier; and
display, by the display system, the target virtual object at a second position, the second position based at least in part on the first position and the movement amount.

### Claim 16 (depends on 15)

The system of claim 15, wherein the indication of the movement of the target virtual object comprises a change in a pose of a user of the wearable device or an input received from a user input device associated with the wearable device.

### Claim 17 (depends on 15)

The system of claim 15, wherein the contextual information comprises a distance from the user to the target virtual object.

### Claim 18 (depends on 17)

The system of claim 17, wherein the multiplier equals to one when the distance is less than a threshold distance, wherein the threshold distance equals to a hand reach of the user.

### Claim 19 (depends on 17)

The system of claim 17, wherein the multiplier increases proportionally with an increase in the distance.

### Claim 20 (depends on 15)

The system of claim 15, wherein the movement comprises one or more of: a position change, a speed, or an acceleration.