Cross reality system with localization service and shared location-based content

A method of operating a portable electronic device to render virtual content in a 3D environment, the method comprising, with one or more processors:

• generating on the portable electronic device a local coordinate frame based on output of one or more sensors on the portable electronic device;
• generating on the portable electronic device information indicative of a location in the 3D environment based on the output of the one or more sensors and an indication of the location in the local coordinate frame;
• sending over a network to a localization service, the information indicative of the location and the indication of the location in the local coordinate frame;
• obtaining from the localization service a transform between a coordinate frame of stored spatial information about the 3D environment and the local coordinate frame;
• obtaining from the localization service one or more data structures, each representing a respective region in the 3D environment and virtual content for display in the respective region;
• obtaining a respective Prism associated with the one or more data structures such that respective virtual content is rendered within the respective Prism;
• and
• rendering, in the respective region of the one or more data structures, within the respective Prism, the virtual content represented in the one or more data structures.

An electronic device configured to operate within a cross reality system, the electronic device comprising:

• one or more sensors configured to capture information about a three-dimensional (3D) environment, the captured information comprising a plurality of images;
• at least one processor;
• at least one computer readable medium storing computer executable instructions that, when executed on a processor of the at least one processor:
• maintain a local coordinate frame for representing positions in the 3D environment based on at least a first portion of the plurality of images;
• manage Prisms associated with one or more applications such that virtual content generated by an application of the one or more applications is rendered within a Prism;
• send over a network, to a service, information derived from outputs of the one or more sensors;
• receive from the service:
• localization information;
• and
• a data structure representing respective virtual content and a region in the 3D environment for rendering the virtual content;
• and
• create a Prism associated with the data structure such that the respective virtual content is rendered within the Prism.