Systems and methods for deep learning-based pedestrian dead reckoning for exteroceptive sensor-enabled devices

A method for estimating six degree of freedom (6DOF) positioning of a computing device while the computing device is in a pedestrian dead reckoning mode by using inertial tracking data and a predictive model trained on both observed exteroceptive sensor data and inertial tracking data, the method comprising:

• detecting, with a computing device, a pedestrian dead reckoning mode triggering condition;
• in response to detecting the pedestrian dead reckoning mode triggering condition, obtaining a set of inertial tracking data from a set of one or more inertial tracking components of the computing device, the set of one or more inertial tracking components including at least one of an accelerometer, a gyroscope and/or a compass;
• obtaining an estimated three degree of freedom (3DOF) velocity of the computing device as output from a predictive model in response to providing at least acceleration data from the accelerometer and angular velocity data obtained by the gyroscope as input into the predictive model, the predictive model having been trained on a set of training data including both observed exteroceptive sensor data and observed inertial tracking data;
• and
• obtaining an estimated 6DOF positioning of the computing device without obtaining exteroceptive sensor data with an exteroceptive sensor system of the computing device by fusing the estimated 3DOF velocity with an estimated 3DOF rotation of the computing device, the estimated 3DOF rotation of the computing device being based on the set of inertial tracking data;
• detecting, with the computing device, an exteroceptive sensor mode triggering condition;
• and
• in response to detecting the exteroceptive sensor mode triggering condition, obtaining radio-based positioning data with a radio-based positioning system of the computing device.

A head-mounted display (HMD) configured for implementing a method for providing training data to an on-device neural network for training a predictive model for use in estimating six degree of freedom (6DOF) positioning of the HMD while the HMD is in a pedestrian dead reckoning mode, the HMD comprising:

• a set of one or more inertial tracking components configured to generate inertial tracking data, the set of one or more inertial tracking components including at least one of an accelerometer, a gyroscope and/or a compass;
• an exteroceptive sensor system configured to generate exteroceptive sensor data;
• one or more processors;
• and
• one or more computer-readable hardware storage devices having stored computer-executable instructions that are operable, when executed by the one or more processors, to cause the system to implement the method for providing training data to an on-device neural network for training a predictive model for use in estimating 6DOF positioning of the HMD while the HMD is in a pedestrian dead reckoning mode, the method comprising:
• obtaining a set of inertial tracking data from the set of one or more inertial tracking components while the computing device is not in the pedestrian dead reckoning mode;
• obtaining the exteroceptive sensor data from the exteroceptive sensor system;
• and
• training a predictive model for use in estimating 6DOF positioning of the HMD when the HMD is in a pedestrian dead reckoning mode by providing training data to an on-device neural network, the training data comprising (1) the set of inertial tracking data as input and (2) at least a portion of the inertial tracking data fused with the exteroceptive data as ground truth output.