# US 2021/0174596 A1 · Cross reality system with simplified programming of virtual content

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0174596 A1
- Document ID: US20210174596A1
- Status: Published application
- Kind: A1
- Application number: 17/115,536
- Filed: 2020-12-08
- Published: 2021-06-10
- Earliest claimed: 2019-12-09
- Assignees: Magic Leap, Inc.
- Applicants: Magic Leap, Inc.
- Inventors: Haiyan Zhang; Robert John Cummings MacDonald
- Web page: https://patentgrep.com/patent/US20210174596A1

## CPC

- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G02B 27/017: {Head mounted}
- G06F 3/04815: Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object
- G06F 3/04817: using icons (graphical or visual programming using iconic symbols G06F8/34)
- G06F 3/0482: Interaction with lists of selectable items, e.g. menus
- G06F 9/542: {Event management; Broadcasting; Multicasting; Notifications}
- G06F 9/547: {Remote procedure calls [RPC]; Web services}
- G06F 16/954: Navigation, e.g. using categorised browsing
- G06F 16/9558: {Details of hyperlinks; Management of linked annotations}
- G02B 2027/0141: {characterised by the informative content of the display}
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A cross reality (XR) system that renders virtual content generated by executing native mode applications may be configured to render web-based content using components that render content from native applications. The system may include a Prism manager that provides Prisms in which content from executing native applications is rendered. For rendering web based content, a browser, accessing the web based content, may be associated with a Prism and may render content into its associated Prism, creating the same immersive experience for the user as when content is generated by a native application. The user may access the web application from the same program launcher menu as native applications. The system may have tools that enable a user to access these capabilities, including by creating for a web location an installable entity that, when processed by the system, results in an icon for the web content in a program launcher menu.

## Claims

### Claim 1

A method of operating a mixed reality system configured to render virtual content generated by executing a native application, wherein the virtual content for the native application is rendered within a prism managed by the system so as to be associated with the native application, the method comprising:
based on an indication to render content from a link, with a browser:
obtaining, with the link, information in a standardized form representing the content; and
based on the obtained information, rendering the content within a prism, wherein the prism is managed by the mixed reality system so as to be associated with the browser, wherein the prism is a volume within which the content is displayed.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
receiving the indication to render the content based on user input selecting an icon associated with the link in a program launcher menu, wherein the program launcher menu comprises at least one icon associated with the native application.

### Claim 3 (depends on 2)

The method of claim 2, wherein:
the mixed reality system comprises a portable electronic device;
the browser is executing on the portable electronic device; and
receiving the indication to render the content further comprises generating a prompt to a user of the portable electronic device to confirm rendering of the content.

### Claim 4 (depends on 2)

The method of claim 2, further comprising:
processing an installable entity, the processing comprising:
generating the icon associated with the link based on the installable entity; and
creating an entry in a registry associating the icon with the link.

### Claim 5 (depends on 4)

The method of claim 4, wherein:
the installable entity comprises a specification of the icon; and
generating the icon comprises rendering the icon based on the specification of the icon.

### Claim 6 (depends on 4)

The method of claim 4, wherein:
the installable entity comprises data specifying a web application; and
generating the icon comprises forming the icon based on the data specifying the web application.

### Claim 7 (depends on 4)

The method of claim 4, further comprising:
generating the installable entity based on user input, the generating comprising a command to install on the mixed reality system a web application associated with the link.

### Claim 8 (depends on 1)

The method of claim 1, wherein rendering the content within the prism comprises:
rendering the content free of navigation controls of the browser.

### Claim 9 (depends on 1)

The method of claim 1, wherein rendering the content within the prism comprises:
rendering 3D content based on a 3D model of the content downloaded from the link.

### Claim 10 (depends on 1)

The method of claim 1, wherein rendering the content within the prism comprises:
generating dynamic 3D content with the browser based on an animated 3D model of the content downloaded from the link.

### Claim 11 (depends on 1)

The method of claim 1, wherein rendering the content within the prism comprises:
at a first time, downloading and caching, on the mixed reality system, information specifying the content; and
at a second time, after the first time, rendering the content based on the cached information.

### Claim 12 (depends on 1)

The method of claim 1, wherein:
the information from the link in the standardized form representing the content is based in part on information obtained from a component of the mixed reality system; and
rendering the content from the link comprises prompting a user of the mixed reality system to authorize access to the component of the mixed reality system.

### Claim 13 (depends on 1)

The method of claim 1, wherein the prism comprises a user interface generated by the mixed reality system and configured to enable a user to control the prism.

### Claim 14

A mixed reality system configured to render virtual content generated by a native application, wherein the virtual content for the native application is rendered within a volume managed by the mixed reality system so as to be associated with the native application, the mixed reality system comprising:
a portable electronic device comprising:
one or more processors;
at least one computer readable medium comprising computer executable instructions that, when executed by a processor of the one or more processors:
based on an indication to render content from a link, with a browser:
obtain information from the link in a standardized form representing the content; and
based on the obtained information, render the content within a volume, wherein the volume is managed by the mixed reality system so as to be associated with the browser.

### Claim 15 (depends on 14)

The mixed reality system of claim 14, wherein the mixed reality system is configured such that the browser has access to hardware components of the portable electronic device.

### Claim 16 (depends on 15)

The mixed reality system of claim 15, wherein the hardware components of the portable electronic device to which the browser has access comprise hardware configured to obtain audio, video, or image data, or any combination thereof.

### Claim 17 (depends on 16)

The mixed reality system of claim 16, wherein the mixed reality system is configured such that the browser conditionally has access to the hardware components of the portable electronic device based on a permission received from a user of the portable electronic device prior to enabling access to the hardware components of the portable electronic device.

### Claim 18 (depends on 14)

The mixed reality system of claim 14, wherein the computer executable instructions, when executed by the processor of the one or more processors:
process an installable entity, the processing comprising:
generating the icon associated with the link based on the installable entity; and
creating an entry in a registry associating the icon with the link.

### Claim 19 (depends on 18)

The mixed reality system of claim 18, wherein:
the installable entity comprises a specification of the icon; and
generating the icon comprises rendering the icon based on the specification of the icon.

### Claim 20 (depends on 18)

The mixed reality system of claim 18, wherein:
the installable entity comprises data specifying a web application; and
generating the icon comprises forming an icon with information identifying the web application.
