US 12,499,618 B2 · Cross reality system with map processing using multi-resolution frame descriptors

Abstract

A distributed, cross reality system efficiently and accurately compares location information that includes image frames. Each of the frames may be represented as a numeric descriptor that enables identification of frames with similar content. The resolution of the descriptors may vary for different computing devices in the distributed system based on degree of ambiguity in image comparisons and/or computing resources for the device. A descriptor computed for a cloud-based component operating on maps of large areas that can result in ambiguous identification of multiple image frames may use high resolution descriptors. High resolution descriptors reduce computationally intensive disambiguation processing. A portable device, which is more likely to operate on smaller maps and less likely to have the computational resources to compute a high resolution descriptor, may use a lower resolution descriptor.

Claims20

Dependent claims

A system for providing shared location-based content, the system comprising:

a portable device, comprising:
at least one processor;
and
at least one computer-readable medium comprising instructions that, when executed, cause the at least one processor to:
render virtual content within a representation of a 3D environment;
compute one or more feature descriptors each representing a respective feature of an image of the 3D environment;
send the one or more feature descriptors to a networked resource;
calculate a first frame descriptor from the one or more feature descriptors, where the first frame descriptor has a first resolution less than a second resolution of a second frame descriptor calculated for the image at the networked resource;
and
localize the portable device by using the first frame descriptor, wherein the localizing is with respect to at least one of (i) a tracking map generated by the portable device, or (ii) at least a portion of a stored map identified by the network resource based on the second frame descriptor.

7 more elements

A system for providing shared location-based content to at least one portable device capable of rendering virtual content in a 3D environment, the system comprising:

one or more processors;
and
at least one computer-readable medium comprising computer executable instructions that, when executed by the one or more processors, cause the one or more processors to:
receive, from a portable electronic device, one or more feature descriptors each representing a respective feature of an image of the 3D environment captured by the portable electronic device;
calculate a second frame descriptor for the image based on the received one or more feature descriptors;
and
localize the portable electronic device to a shared coordinate system based on the second frame descriptor,
wherein the second frame descriptor has a second resolution greater than a first resolution of a first frame descriptor calculated at the at least one portable device.

5 more elements

At least one non-transitory computer-readable medium comprising computer executable instructions that, when executed by one or more processors, cause the one or more processors to:

receive, from a portable device, one or more feature descriptors each representing a respective feature of an image captured by the portable device;
and
calculate a second frame descriptor for the image based on the received one or more feature descriptors;
and
localize the portable device to a shared coordinate system based on the second frame descriptor,
wherein the second frame descriptor has a second resolution greater than a first resolution of a first frame descriptor calculated at the portable device.

3 more elements

Dependencies are inferred from the claim text.

Cross reality system with map processing using multi-resolution frame descriptors

Life of this patent

Cross reality system with map processing using multi-resolution frame descriptors

Life of this patent

Abstract

Claims20