In-vehicle tracking for extended reality
AssigneeSnap Inc.
InventorsJeroen Diederik Hol, Matthias Kalkgruber, Scott Myers, Simon Schreiberhuber
Life of this patent
Estimating the term…
Abstract
Examples disclosed herein describe visual-inertial tracking techniques for extended reality (XR) devices. According to some example methods, an XR device is located in, and movable relative to, a vehicle. The XR device generates device tracking data and accesses vehicle tracking data. The vehicle tracking data is generated by an external sensor configured to measure motion of the vehicle. Consolidated tracking data is generated based on the device tracking data and the vehicle tracking data. In some examples, a pose of the XR device is determined by using the consolidated tracking data.
Claims20
A method performed by an extended reality (XR) device that is located in and movable relative to a vehicle, the XR device including at least one image sensor and at least one inertial sensor, and the method comprising:
- generating device tracking data comprising device image data and device inertial data, the device image data generated using the at least one image sensor of the XR device and the device inertial data generated using the at least one inertial sensor of the XR device;
- detecting an inconsistency between the device image data and the device inertial data;
- based on the detecting of the inconsistency between the device image data and the device inertial data, causing activation of a tracking mode of an external sensor to generate vehicle tracking data;
- accessing the vehicle tracking data generated by the external sensor;
- generating, based on the device tracking data generated by the XR device and the vehicle tracking data generated by the external sensor, consolidated tracking data;
- and
- determining a pose of the XR device based on the consolidated tracking data.
An extended reality (XR) device comprising:
- at least one image sensor
- at least one inertial sensor;
- at least one memory that stores instructions;
- and
- at least one processor configured by the instructions to perform operations comprising, when the XR device is located in and movable relative to a vehicle:
- generating device tracking data comprising device image data and device inertial data, the device image data generated using the at least one image sensor of the XR device and the device inertial data generated using the at least one inertial sensor of the XR device;
- detecting an inconsistency between the device image data and the device inertial data;
- based on the detecting of the inconsistency between the device image data and the device inertial data, causing activation of a tracking mode of an external sensor to generate vehicle tracking data;
- accessing the vehicle tracking data generated by the external sensor;
- generating, based on the device tracking data generated by the XR device and the vehicle tracking data generated by the external sensor, consolidated tracking data;
- and
- determining a pose of the XR device based on the consolidated tracking data.
A non-transitory computer-readable storage medium, the computer-readable storage medium including instructions that when executed by at least one processor of an extended reality (XR) device that is located in and movable relative to a vehicle, cause the at least one processor to perform operations comprising:
- generating device tracking data, the XR device including at least one image sensor and at least one inertial sensor, and the device tracking data comprising device image data and device inertial data, the device image data generated using the at least one image sensor of the XR device and the device inertial data generated using the at least one inertial sensor of the XR device;
- detecting an inconsistency between the device image data and the device inertial data;
- based on the detecting of the inconsistency between the device image data and the device inertial data, causing activation of a tracking mode of an external sensor to generate vehicle tracking data;
- accessing the vehicle tracking data generated by the external sensor;
- generating, based on the device tracking data generated by the XR device and the vehicle tracking data generated by the external sensor, consolidated tracking data;
- and
- determining a pose of the XR device based on the consolidated tracking data.
Dependencies are inferred from the claim text.