Back to results
US 10,769,440 B1Granted
Visual-inertial positional awareness for autonomous and non-autonomous tracking
AssigneeTrifo, Inc.
InventorsZhe Zhang, Grace Tsai, Shaoshan Liu
Life of this patent
Estimating the term…
Abstract
The described positional awareness techniques employing visual-inertial sensory data gathering and analysis hardware with reference to specific example implementations implement improvements in the use of sensors, techniques and hardware design that can enable specific embodiments to provide positional awareness to machines with improved speed and accuracy.
Claims20
A system including:
- two or more mobile autonomous units, including a first autonomous unit and a second autonomous unit, each having a mobile platform and disposed thereon:
- a visual sensor comprising cameras providing capturing images including at least two frames, thereby providing a 360-degrees view about a centerline of the mobile platform;
- and at least one of:
- a multi-axis inertial measuring unit (IMU) sensor capable of providing measurement of at least acceleration using one or more accelerometers;
- and
- a global positioning system (GPS) receiver;
- and
- a map server, including a processor and a coupled memory storing instructions to build 3D maps of surrounding scenery encountered by the autonomous units using information sourced by the autonomous units, which instructions when executed by the processor perform:
- receiving a set of keyrigs from a first autonomous unit, each keyrig comprising visual information of surrounding scenery captured by the visual sensor of the first autonomous unit and a position of the first autonomous unit where the visual information was captured, the position of the autonomous unit generated using combinations of global positioning system (GPS) receiver, the multi-axis inertial measurement unit (IMU), and visual information of the surrounding scenery by the first autonomous unit during travel from a starting point to an end point;
- classifying at least one of one or more objects from the visual information of the surrounding scenery from the set of keyrigs into a set of moving objects and a set of non-moving objects;
- selecting a subset of keyrig from the set of keyrigs and determining a sparse 3D mapping of object feature points taken from the visual information of the surrounding scenery from the subset of keyrigs for the set of non-moving objects;
- building a 3D map of object feature points from the sparse 3D mapping of object feature points using visual information of the surrounding scenery from the set of keyrigs for the set of non-moving objects, wherein the 3D map is accurate within 20 centimeters;
- and
- providing the 3D map via a communications link to one or more additional autonomous units to guide the one or more additional autonomous units at a future time.
11 more elements
A method for building 3D maps from a surrounding scenery using information sourced by one or more moving autonomous units that include at least a camera visual sensor and at least one selected from a global positioning system and an inertial measurement unit, including:
- receiving a set of keyrigs from a first autonomous unit, each keyrig comprising visual information of the surrounding scenery and a position of the first autonomous unit where the visual information was captured, the position of the autonomous unit generated using combinations of global positioning system, inertial measurement unit, and visual information of the surrounding scenery by the first autonomous unit during travel from a starting point to an end point;
- classifying at least one of one or more objects from the visual information of the surrounding scenery from the set of keyrigs into a set of moving objects and a set of non-moving objects;
- selecting a subset of keyrig from the set of keyrigs and determining a sparse 3D mapping of object feature points taken from the visual information of the surrounding scenery from the subset of keyrigs for the set of non-moving objects;
- building a 3D map of object feature points from the sparse 3D mapping of object feature points using visual information of the surrounding scenery from the set of keyrigs for the set of non-moving objects, wherein the 3D map is accurate within 20 centimeters;
- and
- providing the 3D map via a communications link to one or more additional autonomous units to guide the one or more additional autonomous units at a future time.
3 more elements
A non-transitory computer readable medium storing instructions for building 3D maps from surrounding scenery using information sourced by one or more moving autonomous units, which instructions when executed by a processor perform:
- receiving a set of keyrigs from a first autonomous unit, each keyrig comprising visual information from surrounding scenery and a position of the first autonomous unit where the visual information was captured, the position of the autonomous unit generated using combinations of global positioning system, inertial measurement unit, and visual information of the surrounding scenery by the first autonomous unit during travel from a starting point to an end point;
- classifying at least one of one or more objects from the visual information of the surrounding scenery from the set of keyrigs into a set of moving objects and a set of non-moving objects;
- selecting a subset of keyrig from the set of keyrigs and determining a sparse 3D mapping of object feature points taken from the visual information of the surrounding scenery from the subset of keyrigs for the set of non-moving objects;
- building a 3D map of object feature points from the sparse 3D mapping of object feature points using visual information of the surrounding scenery from the set of keyrigs for the set of non-moving objects, wherein the 3D map is accurate within 20 centimeters;
- and
- providing the 3D map via a communications link to one or more additional autonomous units to guide the one or more additional autonomous units at a future time.
3 more elements
A method for building 3D maps from a surrounding scenery using information sourced by one or more moving autonomous units that include at least a camera visual sensor and at least one selected from a global positioning system and an inertial measurement unit, including:
- receiving a set of keyrigs from a first autonomous unit, each keyrig comprising visual information of the surrounding scenery and a position of the first autonomous unit where the visual information was captured, the position of the autonomous unit generated using combinations of global positioning system, inertial measurement unit, and visual information of the surrounding scenery by the first autonomous unit during travel from a starting point to an end point;
- classifying at least one of one or more objects from the visual information of the surrounding scenery from the set of keyrigs into a set of moving objects and a set of non-moving objects;
- selecting a subset of keyrig from the set of keyrigs and determining a sparse 3D mapping of object feature points taken from the visual information of the surrounding scenery from the subset of keyrigs for the set of non-moving objects;
- building a 3D map of object feature points from the sparse 3D mapping of object feature points using visual information of the surrounding scenery from the set of keyrigs for the set of non-moving objects, wherein the 3D map is accurate within 20 centimeters;
- and
- providing the 3D map via a communications link to a map server that stores and distributes 3D maps to guide the autonomous unit at a future time and at least one other autonomous unit.
3 more elements
Dependencies are inferred from the claim text.