Back to results
US 2026/0057539 A1Published application
System and method for box segmentation and measurement
Assignee4DMobile, LLC
InventorsMatthew D. Miller, Josh Berry, Mark Boyer
Life of this patent
Estimating the term…
Abstract
A mobile device is capable of being carried by a user and directed at a target object. The mobile device may implement a system to dimension the target object. The system, by way of the mobile device, may image the target object to and receive a 3D image stream, including one or more frames. Each frame may include a plurality of points, where each point has an associated depth value. Based on the depth value of the plurality of points, the system, by way of the mobile device, may determine one or more dimensions of the target object.
Claims20
A volume dimensioning system comprising:
- an image sensor configured to capture imaging data associated with a payload object positioned on a pallet, the imaging data comprising a sequence of frames, each frame comprising a depth map with two-dimensional (2D) pixel coordinates and depth values;
- and
- one or more processors in communication with the image sensor, the one or more processors configured to:
- annotate the imaging data to output a first predicted bounding box and a second predicted bounding box, wherein the first predicted bounding box is for the payload object and the second predicted bounding box is for the pallet;
- deproject the depth map into three-dimensional space to generate a point cloud;
- remove a ground plane from the point cloud;
- fit a line to the depth map along a bottom of the pallet;
- perform spatial clustering on the point cloud to segment the payload object and the pallet from a background within the point cloud, wherein the spatial clustering is performed in response to removing the ground plane from the point cloud, wherein seeds for the spatial clustering are derived from locations of the first predicted bounding box, the second predicted bounding box, and the line;
- project the point cloud onto a reference plane to generate a point projection in response to performing the spatial clustering, wherein the one or more processors determine the reference plane as being orthogonal to the ground plane and passing through the line;
- calculate an oriented bounding box from the point projection;
- and
- determine dimensions of the payload object and the pallet based on the oriented bounding box.
9 more elements
A volume dimensioning system comprising:
- an image sensor configured to capture imaging data associated with a target object, the imaging data comprising a sequence of frames, each frame comprising a depth map with two-dimensional (2D) pixel coordinates and depth values;
- and
- one or more processors in communication with the image sensor, the one or more processors configured to:
- deproject the depth map into three-dimensional space to generate a point cloud;
- remove a ground plane from the point cloud;
- perform spatial clustering on the point cloud to segment the target object from a background within the point cloud, wherein the spatial clustering is performed in response to removing the ground plane from the point cloud, wherein the spatial clustering isolates point cluster in the point cloud corresponding to the target object;
- project the point cluster onto the ground plane and a top plane to generate a horizontal plane projection;
- calculate an oriented bounding box from the horizontal plane projection;
- and
- determine dimensions of the target object based on the oriented bounding box.
7 more elements
A volume dimensioning system comprising:
- an image sensor configured to capture imaging data associated with a target object, the imaging data comprising a sequence of frames, each frame comprising a depth map with two-dimensional (2D) pixel coordinates and depth values;
- and
- one or more processors in communication with the image sensor, the one or more processors configured to:
- deproject the depth map into three-dimensional space to generate a point cloud;
- identify an origin point within the depth values, wherein the origin point is associated with a top corner of the target object, wherein the origin point is a local minimum of the depth values within a cursor of the imaging data, wherein the one or more processors are configured to examine the depth values for the local minimum to identify the origin point;
- refine the origin point using color values of the imaging data by a color image-based primary point detection model;
- crawl from the origin point along a first edge to a first corner, along a second edge to a second corner, and along a third edge to a third corner of the target object to detect the first edge, the first corner, the second edge, the second corner, the third edge, and the third corner;
- deproject the depth map into three-dimensional (3D) points;
- remove a ground plane from the point cloud;
- perform spatial clustering on the point cloud to segment the target object from a background within the point cloud in response to removing the ground plane;
- project the point cloud onto a reference plane to generate a point projection;
- and
- examine a convex hull of the point projection for bottom keypoints.
10 more elements
Dependencies are inferred from the claim text.