Vision sensing device and method

A vision sensing method comprising:

• projecting, with a laser unit, a laser pattern within a field of view of a camera arranged in a device;
• capturing inertial data from an inertial measurement unit arranged in the device as a user moves the device;
• capturing visual data from the field of view with the camera as the user moves the device;
• capturing depth data with the laser unit as the user moves the device;
• and
• generating, with at least one processor, an RGB-D point cloud based on the visual data, the inertial data, and the depth data by:
• generating a plurality of colored laser points based on a colored light in the visual data and the depth data;
• determining an estimated camera motion;
• mapping each colored laser point of the plurality of colored laser points into the RGB-D point cloud within a spatial representation;
• and
• aligning at least a subset of the plurality of colored laser points based on the estimated camera motion.

A computer program product for vision sensing comprising at least one non-transitory computer-readable medium including program instructions that, when executed by at least one processor, cause the at least one processor to:

• project, with a laser unit, a laser pattern within a field of view of a camera arranged in a device;
• capture inertial data from an inertial measurement unit arranged in the device as a user moves the device;
• capture visual data from the field of view with the camera as the user moves the device;
• capture depth data with the laser unit as the user moves the device;
• and
• generate, with at least one processor, an RGB-D point cloud based on the visual data, the inertial data, and the depth data by:
• generating a plurality of colored laser points based on a colored light in the visual data and the depth data;
• determining an estimated camera motion;
• mapping each colored laser point of the plurality of colored laser points into the RGB-D point cloud within a spatial representation;
• and
• aligning at least a subset of the plurality of colored laser points based on the estimated camera motion.

A vision sensing method comprising:

• projecting, with a laser unit, a laser pattern within a field of view of a camera arranged in a device;
• capturing inertial data from an inertial measurement unit arranged in the device as a user moves the device;
• capturing visual data from the field of view with the camera as the user moves the device, wherein the camera is configured to capture the visual data by:
• capturing a plurality of images in a series;
• and
• varying an exposure time of the camera while capturing at least two sequential images of the plurality of images in the series, such that the visual data comprises at least one first image with a first exposure time and at least one second image with a second exposure time longer than the first exposure time, wherein the plurality of images comprises a first subset of images captured with the first exposure time and a second subset of images captured with the second exposure time;
• capturing depth data with the laser unit as the user moves the device;
• and
• generating, with at least one processor, an RGB-D point cloud based on the visual data, the inertial data, and the depth data by:
• tracking a plurality of visual features in each image of the second subset of images;
• determining a subset of visual features of the plurality of features based on a distance of each visual feature of the subset of visual features to the laser pattern;
• and
• determining a subset of keyframes from the second subset of images based on the subset of visual features.