Cross reality system for large scale environments

An XR system that supports specification of a position of virtual content relative to persisted maps in a database of persisted maps, the XR system comprising

• a localization service configured to receive from a portable electronic device information about a plurality of collections of features in images of a three-dimensional (3D) environment, the information comprising positions for the features of the plurality of collections of features expressed in a coordinate frame, wherein an estimated direction of gravity is sent to the portable electronic device,
• wherein the localization service comprises at least one processor configured to execute computer-executable instructions, the computer-executable instructions implementing a localization component, the localization component configured to:
• compute a plurality of candidate localizations for the plurality of collections of features by, for each of the plurality of collections of features, performing a process to determine as a candidate localization a transformation between the collection of features and a portion of a persisted map in the database of persisted maps, wherein:
• the portion of the persisted map has an associated estimated direction of gravity;
• and
• the process of determining is constrained to determine transformations that align the estimated direction of gravity for the collection of features with the associated estimated direction of gravity of the portion of the persisted map;
• and
• generate a localization of the portable electronic device based on consensus among the plurality of candidate localizations,
• wherein the process of determining the transformations of the candidate localizations is constrained by:
• fixing two rotational degrees of freedom based, at least in part, on an orientation of the respective coordinate frame with respect to the estimated direction of gravity, and
• computing one rotational degree of freedom and three translational degrees of freedom based, at least in part, on the fixed two rotational degrees of freedom.

An electronic device configured to operate within a cross reality system, the electronic device comprising:

• one or more sensors configured to capture information about a three-dimensional (3D) environment, the captured information comprising a plurality of images;
• and
• at least one processor configured to execute computer executable instructions, wherein the computer executable instructions comprise instructions for:
• extracting a plurality of collections of features from the plurality of images of the 3D environment;
• determining an estimated direction of gravity with respect to a coordinate frame local to the electronic device;
• expressing positions of the features of the plurality of collections of features in the coordinate frame;
• sending to a localization service of the cross reality system information about the plurality of collections of features, the information indicating the positions of the features of the plurality of collections of features and the estimated direction of gravity with respect to the coordinate frame;
• and
• receiving from the localization service a pose of the electronic device with respect to a persisted map in a database of persisted maps, wherein determining the pose of the electronic device is constrained by:
• fixing two rotational degrees of freedom based, at least in part, on an orientation of the respective coordinate frame with respect to the estimated direction of gravity, and
• computing one rotational degree of freedom and three translational degrees of freedom based, at least in part, on the fixed two rotational degrees of freedom such that the pose of the electronic device is received in less than 6 seconds after the sending to the localization service.