Geometry modeling of eyewear devices with flexible frames
AssigneeSnap Inc.
InventorsClemens Birklbauer, Georg Halmetschlager-Funek, Jeroen Hol, Matthias Kalkgruber, Daniel Wagner
Life of this patent
Estimating the term…
Abstract
An eyewear device with flexible frame for Augmented Reality (AR) is disclosed. At least two sensors and a display are mounted on the flexible frame. When in use, the real time geometry of the eyewear device may change from factory calibrated geometry, resulting in low quality AR rendering. A modeling module is provided to model the real time geometry of the eyewear device on the fly using sensor information of the at least two sensors. The modeled real time geometry is then provided to a rendering module to accurately display the AR to the user.
Claims19
An eyewear device for displaying augmented reality images, comprising:
- a sensor;
- at least one image sensor;
- at least one display for displaying the augmented reality images, wherein the eyewear device has a predetermined geometry defining spatial relations of at least two of the sensor, the at least one image sensor, and the at least one display;
- a rendering module;
- and
- an augmented reality image rendering system including a processor that executes instructions to perform operations including:
- tracking poses of the at least one image sensor and the sensor,
- detecting whether the predetermined geometry as calibrated in factory is currently valid or invalid,
- when the predetermined geometry as calibrated in factory is detected to be invalid, modeling a change in the predetermined geometry to generate updated spatial information of the changed predetermined geometry,
- providing the updated spatial information of the changed predetermined geometry to the rendering module for rendering of the augmented reality images including virtual content and real-world objects on the at least one display in accordance with the changed predetermined geometry, and
- computing, using a computer vision processing algorithm of a motion tracking module, poses of the at least one image sensor and the sensor from the updated spatial information.
A method for displaying augmented reality images on an eyewear device including a sensor, at least one image sensor, and at least one display for displaying the augmented reality images, the eyewear device having a predetermined geometry defining spatial relations of at least two of the sensor, the at least one image sensor, and the at least one display, comprising:
- tracking poses of the at least one image sensor and the sensor,
- detecting whether the predetermined geometry as calibrated in factory is currently valid or invalid,
- when the predetermined geometry as calibrated in factory is detected to be invalid, modeling a change in the predetermined geometry to generate updated spatial information of the changed predetermined geometry,
- providing the updated spatial information of the changed predetermined geometry to a rendering module,
- rendering, by the rendering module, the augmented reality images including virtual content and real-world objects on the at least one display in accordance with the changed predetermined geometry, and
- computing poses of the at least one image sensor and the sensor from the updated spatial information.
A non-transitory computer-readable medium comprising instructions stored therein that, when executed by one or more processors, cause the one or more processors to display augmented reality images on an eyewear device including a sensor, at least one image sensor, and at least one display for displaying the augmented reality images, the eyewear device having a predetermined geometry defining spatial relations of at least two of the sensor, the at least one image sensor, or the at least one display, by performing operations including:
- tracking poses of the at least one image sensor and the sensor,
- detecting whether the predetermined geometry as calibrated in factory is currently valid or invalid,
- when the predetermined geometry as calibrated in factory is detected to be invalid, modeling a change in the predetermined geometry to generate updated spatial information of the changed predetermined geometry,
- providing the updated spatial information of the changed predetermined geometry to a rendering module,
- rendering, by the rendering module, the augmented reality images including virtual content and real-world objects on the at least one display in accordance with the changed predetermined geometry, and
- computing poses of the at least one image sensor and the sensor from the updated spatial information.
Dependencies are inferred from the claim text.