Cross reality system

An electronic system for maintaining persistent spatial information about a 3D environment for rendering virtual content on each of a plurality of portable devices, comprising:

• a networked computing device comprising:
• at least one processor;
• at least one storage device connected to the processor;
• map storing routine, executable with the at least one processor, to receive from portable devices of the plurality of portable devices, a plurality of maps and store map information on the at least one storage device, wherein each of the plurality of received maps comprises at least one coordinate frame;
• and
• a map transmitter, executable with the at least one processor, to:
• receive location information from a portable device of the plurality of portable devices;
• select one or more maps from among the stored maps;
• transmit to the portable device of the plurality of portable devices information from the selected one or more maps, wherein the transmitted information comprises a coordinate frame of a map of the selected one or more maps.

A portable device, comprising:

• one or more sensors configured to capture information about a three-dimensional (3D) environment, the captured information comprising a plurality of images;
• and
• at least one processor configured to execute computer executable instructions to generate a map of at least a portion of the 3D environment based on the plurality of images, wherein the computer executable instructions further comprise instructions for:
• maintaining on the portable device a local coordinate frame based on output of the one or more sensors;
• obtaining a stored coordinate frame from stored spatial information about the 3D environment;
• computing a transformation between the local coordinate frame and the obtained stored coordinate frame;
• receiving a specification of a virtual object having a virtual object coordinate frame and a location of the virtual object with respect to the obtained stored coordinate frame;
• and
• rendering the virtual object on a display of the portable device at a location determined, at least in part, based on the computed transformation and the received location of the virtual object.