# US 2023/0267697 A1 · Object creation with physical manipulation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0267697 A1
- Document ID: US20230267697A1
- Status: Published application
- Kind: A1
- Application number: 18/182,590
- Filed: 2023-03-13
- Published: 2023-08-24
- Earliest claimed: 2018-02-22
- Assignees: Magic Leap, Inc.
- Applicants: Magic Leap, Inc.
- Inventors: Tim ZURMOEHLE; Andrea Isabel MONTOYA; Robert John Cummings MACDONALD; Sakina GROTH; Genevieve MAK
- Web page: https://patentgrep.com/patent/US20230267697A1

## CPC

- G06F 1/1694: {the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer}
- G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/016: {Input arrangements with force or tactile feedback as computer generated output to the user}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/0346: with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
- G06F 16/9566: {URL specific, e.g. using aliases, detecting broken or misspelled links}
- G06F 16/957: Browsing optimisation, e.g. caching or content distillation
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06T 2200/24: involving graphical user interfaces [GUIs]
- G06T 2219/2004: Aligning objects, relative positioning of parts
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Described are improved systems and methods for navigation and manipulation of interactable objects in a 3D mixed reality environment. Improved systems and methods are provided to implement physical manipulation for creation and placement of interactable objects, such as browser windows and wall hangings. A method includes receiving data indicating a selection of an interactable object contained within a first prism at the start of a user interaction. The method also includes receiving data indicating an end of the user interaction with the interactable object. The method further includes receiving data indicating a physical movement of the user corresponding to removing the interactable object from the first prism between the start and the end of the user interaction. Moreover, the method includes creating a second prism to contain the data associated with the interactable object at the end of the user interaction with the interactable object.

## Claims

### Claim 1

A system, comprising:
a mixed reality display device configured for displaying an interactable object as virtual content at a first location within a spatially organized three-dimensional environment to a user;
a user interface configured for detecting interaction of the user with the interactable object; and
a processor configured for:
receiving data indicating a selection of the interactable object contained within a first prism at a start of a user interaction, wherein the data indicating the selection of the interactable object corresponds to at least a portion of a body of the user interacting with a link;
generating a preview of the link by accessing content for a Uniform Resource Locator (URL) associated with the link;
instructing the mixed reality display device to display the preview including at least a portion of the content for the URL associated with the link;
receiving data indicating an end of the user interaction with the interactable object;
receiving data indicating a physical movement of the user corresponding to removing the interactable object from the first prism between the start and the end of the user interaction, wherein the physical movement of the user corresponds to a movement of the preview such that visual display of the preview moves during the user interaction according to the physical movement; and
creating a second prism to contain the interactable object at the end of the user interaction with the interactable object.

### Claim 2 (depends on 1)

The system of claim 1, wherein the processor is also configured for instructing the mixed reality display device to display the preview at a location of the interactable object between the start and the end of the user interaction when the interactable object is located outside of bounds of the first prism.

### Claim 3 (depends on 1)

The system of claim 1, further comprising a user input device configured to provide the data indicating the selection of the interactable object and the data indicating the end of the user interaction with the interactable object.

### Claim 4 (depends on 3)

The system of claim 3, wherein the user input device is a haptics controller.

### Claim 5 (depends on 4)

The system of claim 4, wherein the haptics controller is a totem device having at least six degrees of freedom.

### Claim 6 (depends on 4)

The system of claim 4, further comprising a processing system associated with the haptics controller,
wherein the processing system is configured to translate the physical movement of the user into event objects, and
wherein the event objects correlate to data for movement or positioning of the haptics controller.

### Claim 7 (depends on 4)

The system of claim 4, wherein the haptics controller comprises a trigger, and
wherein a location of the interactable object at the end of the user interaction is determined by identifying a location at which the trigger is released after the selection of the interactable object.

### Claim 8 (depends on 3)

The system of claim 3, wherein the user input device is a camera to detect a user gesture.

### Claim 9 (depends on 8)

The system of claim 8, wherein the data indicating the selection of the interactable object corresponds to a selection gesture.

### Claim 10 (depends on 8)

The system of claim 8, wherein the data indicating the end of the user interaction with the interactable object corresponds to a release gesture.

### Claim 11 (depends on 1)

The system of claim 1, wherein the interactable object is both opened and placed with the user interaction.

### Claim 12 (depends on 1)

The system of claim 1, wherein the processor is also configured for determining an orientation for the interactable object by:
determining a surface normal of a location of the interactable object at the end of the user interaction;
determining an orientation data of the interactable object; and
aligning the surface normal of the location with the orientation data of the interactable object.

### Claim 13 (depends on 1)

The system of claim 1, wherein the interactable object comprises a wall hanging.

### Claim 14

A system, comprising:
a mixed reality display device configured for displaying an interactable object as virtual content at a first location within a spatially organized three-dimensional environment to a user;
a user interface configured for detecting interaction of the user with the interactable object; and
a processor configured for:
receiving data indicating a selection of the interactable object at a start of a user interaction, wherein the data indicating the selection of the interactable object corresponds to at least a portion of a body of the user interacting with a link;
generating a preview of the link by accessing content for a Uniform Resource Locator (URL) associated with the link;
displaying the preview including at least a portion of the content for the URL associated with the link;
receiving data indicating an end of the user interaction with the interactable object;
determining a location of the interactable object at the end of the user interaction;
receiving data indicating a physical movement of the user, wherein the physical movement of the user corresponds to a movement of the preview such that visual display of the preview moves during the user interaction according to the physical movement; and
creating a new prism for the interactable object at the end of the user interaction when the location of the interactable object at the end of the user interaction is outside of an initial prism of the interactable object.

### Claim 15 (depends on 14)

The system of claim 14, further comprising a user input device configured to provide the data indicating the selection of the interactable object and the data indicating the end of the user interaction with the interactable object.

### Claim 16 (depends on 15)

The system of claim 15, wherein the user input device is a haptics controller.

### Claim 17 (depends on 16)

The system of claim 16, further comprising a processing system associated with the haptics controller,
wherein the processing system is configured to translate the physical movement of the user into event objects, and
wherein the event objects correlate to data for movement or positioning of the haptics controller.

### Claim 18

A system, comprising:
a mixed reality display device configured for displaying an interactable object as virtual content at a first location within a spatially organized three-dimensional environment to a user;
a user interface configured for detecting interaction of the user with the interactable object; and
a processor configured for:
receiving data indicating the selection of the interactable object contained within a first prism at a start of a user interaction, wherein the data indicating the selection of the interactable object corresponds to at least a portion of a body of the user interacting with a link;
generating a preview of the link by accessing content for a Uniform Resource Locator (URL) associated with the link;
displaying the preview including at least a portion of the content for the URL associated with the link;
receiving data indicating a physical movement of the user, wherein the physical movement of the user corresponds to a movement of the preview such that visual display of the preview moves during the user interaction according to the physical movement; and
creating a second prism at a location of the interactable object at an end of the user interaction to contain the interactable object when the data indicating the physical movement of the user indicates that the interactable object is outside of the first prism at the end of the user interaction.

### Claim 19 (depends on 18)

The system of claim 18, further comprising a user input device configured to provide the data indicating the selection of the interactable object and the data indicating the end of the user interaction with the interactable object,
wherein the user input device is a haptics controller.

### Claim 20 (depends on 19)

The system of claim 19, further comprising a processing system associated with the haptics controller,
wherein the processing system is configured to translate the physical movement of the user into event objects, and
wherein the event objects correlate to data for movement or positioning of the haptics controller.