Visual-inertial positional awareness for autonomous and non-autonomous tracking
AssigneeTrifo, Inc.
InventorsZhe Zhang, Grace Tsai, Shaoshan Liu
Life of this patent
Estimating the term…
Abstract
The described positional awareness techniques employing visual-inertial sensory data gathering and analysis hardware with reference to specific example implementations implement improvements in the use of sensors, techniques and hardware design that can enable specific embodiments to provide positional awareness to machines with improved speed and accuracy.
Claims21
A system including:
- a mobile platform having disposed thereon:
- at least one camera;
- an inertial measuring unit (IMU) comprising one or more of accelerometers configured to measure linear acceleration and gyroscopes configured to measure rotational rate;
- and
- an interface to a host including one or more processors coupled to memory, the memory loaded with computer instructions to update a position of the mobile platform that includes the camera and the IMU, the instructions, when executed on the processors, implement actions comprising:
- receiving a location of the mobile platform and a perspective, including view direction, of the camera, referred to collectively as an initial pose;
- while waiting for a new frame, updating the initial pose, using inertial data from the IMU, thereby generating a propagated pose;
- retrieving at least some feature points within a field of view of the propagated pose from a 3D map;
- extracting new features from the new frame;
- matching the extracted new features to the retrieved feature points, producing a set of matching features;
- and
- determining a perspective from which the new frame was viewed by the camera from a visually corrected pose calculated using positions of the matching features in the set of matching features;
- and
- responsive to requests for location of the mobile platform, and/or the perspective of the camera, providing data based on at least one of the propagated pose and the visually corrected pose.
A method of updating a position of a mobile unit that includes a camera and an inertial measuring unit (IMU) comprising one or more of accelerometers configured to measure linear acceleration and gyroscopes configured to measure rotational rate, the method including:
- receiving a location of the mobile unit and perspective, including view direction, of the camera, referred to collectively as an initial pose;
- while waiting for a new frame, updating the initial pose using inertial data from the IMU, thereby generating a propagated pose;
- retrieving at least some feature points within a field of view of the propagated pose from a 3D map;
- extracting new features from the new frame;
- matching the extracted new features to the retrieved feature points, producing a set of matching features;
- and
- determining a perspective from which the new frame was viewed by the camera from a visually corrected pose calculated using positions of the matching features in the set of matching features;
- and
- responsive to requests for location of the mobile unit, and/or the perspective of the camera, providing data based on at least one of the propagated pose and the visually corrected pose.
A non-transitory computer readable storage medium impressed with computer program instructions to update a position of a mobile unit that includes a camera and an inertial measuring unit (IMU) comprising one or more of accelerometers configured to measure linear acceleration and gyroscopes configured to measure rotational rate, the instructions, when executed on a processor, implement a method comprising:
- receiving a location of a mobile unit and perspective, including view direction, of the camera, referred to collectively as an initial pose;
- while waiting for a new frame, updating the initial pose using inertial data from the IMU, thereby generating a propagated pose;
- retrieving at least some feature points within a field of view of the propagated pose from a 3D map;
- extracting new features from the new frame;
- matching the extracted new features to the retrieved feature points, producing a set of matching features;
- and
- determining a perspective from which the new frame was viewed by the camera from a visually corrected pose calculated using positions of the matching features in the set of matching features;
- and
- responsive to requests for location of the mobile unit, and/or the perspective of the camera, providing data based on at least one of the propagated pose and the visually corrected pose.
Dependencies are inferred from the claim text.