# US 11,908,151 B2 · System and method for mobile 3D scanning and measurement > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,908,151 B2 - Document ID: US11908151B2 - Status: Granted - Kind: B2 - Application number: 18/187,730 - Filed: 2023-03-22 - Published: 2023-07-20 - Granted: 2024-02-20 - Earliest claimed: 2019-11-28 - Assignees: Xesto Inc. - Applicants: Xesto Inc. - Inventors: Mehmet Afiny Affan Akdemir; Christian Garcia Salguero; Victoria Sophie Howe - Web page: https://patentgrep.com/patent/US11908151B2 ## CPC - G06T 7/593: from stereo images - G06T 17/00: Three-dimensional [3D] modelling for computer graphics - G06T 7/33: using feature-based methods - G06T 7/579: from motion - G06T 19/00: Manipulating three-dimensional [3D] models or images for computer graphics - G06T 2207/10028: Range image; Depth image; 3D point clouds - G06T 2207/20036: Morphological image processing - G06T 2207/20081: Training; Learning - G06T 2207/20084: Artificial neural networks [ANN] - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract Systems and methods are provided for three-dimensional scanning and measurement by a device having a processor. The processor is configured to receive images of an object from at least. two angles; preprocess the images using morphological refinement; create a source point cloud based on the images; remove outliers from the source point cloud; globally register the source point cloud to generate a transformed source point cloud; compare the transformed source point cloud with a target point cloud to generate a stitched point cloud that thereby creates a stitched 3D model; measure the resulting stitched 3D model; and provide the resulting stitched 3D model for comparison to a digitized item to assess sizing of the 3D model to the item. ## Claims ### Claim 1 A system of deformable object stitching, comprising: a mobile device to scan a plurality of images of an object, the plurality of images providing views of the object from at least two angles; at least one processor coupled to the mobile device to process the plurality of images of the object, the plurality of images providing views of the object from at least two angles comprising a first point cloud and a second point cloud; a non-transient computer-readable storage medium having stored thereon computer-executable instructions that, when executed by the at least one processor, cause the system to: receive the first point cloud and the second point cloud having a partial overlap; find first prominent features in a subregion G1 and a subregion G2 in a region of the partial overlap; determine first geodesics based on the first prominent features; determine second prominent features in a subregion H1 and a subregion H2 in the region of the partial overlap; determine second geodesics based on the second prominent features; generate a first correspondence between the subregion G1, and the subregion H1; generate a second correspondence between the subregion G2 and the subregion Hz; deform the first geodesics and the second geodesics based on the first correspondence and the second correspondence; stitch the first point cloud and the second point cloud, whereby a stitched point cloud is generated; calculate a required deformation to transform the first geodesics to the second geodesics; and apply the required deformation to the second point cloud to match the second point cloud to the first point cloud, whereby a refined stitched point cloud is provided. ### Claim 2 (depends on 1) The system of claim 1, wherein the instructions to find the first prominent features comprise instructions to: estimate a curvature in the subregion G1 or the subregion G2; and then refine the estimated curvature by an iterative calculation of the curvature over one or more iterations, such that a region of high curvature is re-calculated by considering a smaller neighborhood in the region of high curvature; calculate the curvature based on the smaller neighborhood; and repeat the iterations until sampling in the smaller neighborhood does not increase the curvature in the region of high curvature by more than a specified threshold. ### Claim 3 (depends on 1) The system of claim 1, wherein the instructions to find the second prominent features comprise instructions to: estimate a curvature in the subregion H1 or the subregion H2; then refine the estimated curvature by an iterative calculation of the curvature over one or more iterations, such that a region of high curvature is re-calculated by considering a smaller neighborhood in the region of high curvature; calculate the curvature based on the smaller neighborhood; and repeat the iterations until sampling in the smaller neighborhood does not increase the curvature in the region of high curvature by more than a specified threshold. ### Claim 4 (depends on 1) The system of claim 1, wherein the instructions to find the first prominent features comprise instructions to: approximate a curvature on the subregion G1 and the subregion G2 by discretizing a curvature operator; and extract the first prominent features with respect to the curvature where the curvature is high or zero, a high curvature representing sharp corners and a zero curvature representing a flat region. ### Claim 5 (depends on 1) The system of claim 1, wherein the instructions to find the second prominent features comprise instructions to approximate a curvature on the subregion H1 and the subregion H2 by: a discretization of a curvature operator; and an extraction of the second prominent features with respect to the curvature where the curvature is high or zero, a high curvature representing sharp corners and a zero curvature representing a flat region. ### Claim 6 (depends on 1) The system of claim 1, wherein the instructions to determine the first geodesics and the instructions determine the second geodesics comprise instructions to: triangulate on the respective subregions; and determine a shortest path between two points on a triangulated surface. ### Claim 7 (depends on 1) The system of claim 1, wherein the instructions to generate the first correspondence and the instructions generate the second correspondence comprise instructions to: rank the first prominent features and the second prominent features, respectively, by curvature in the first point cloud and the second point cloud, respectively; and match points between the first point cloud and the second point cloud. ### Claim 8 (depends on 1) The system of claim 1, wherein the calculate the required deformation comprise instructions to: produce a first surface determined by connecting any two points between subregion G1 and subregion H1, produce a geodesic, and extend the geodesic into a second surface by considering a neighborhood about the geodesic. ### Claim 9 (depends on 1) The system of claim 1, wherein the instructions deform the first geodesics and the second geodesics further comprise instructions to: estimate a first fundamental form of a surface by finite difference methods; and deform the surface by an As Rigid As Possible (ARAP) deformation using the first correspondence and the second correspondence. ### Claim 10 (depends on 9) The system of claim 9, wherein the instructions deform the first geodesics and the second geodesics further comprise instructions to: determine a similarity between deformed subregions by an estimate of a second fundamental form for each of the subregions; and a comparison of a coefficient between the first fundamental form and the second fundamental form. ### Claim 11 (depends on 1) The system of claim 1, wherein the instructions cause the system to: extract measurements from the refined stitched point cloud, with an employment of a geodesic distance function. ### Claim 12 A computer-implemented method of deformable object stitching, comprising: receiving a first point cloud and a second point cloud having a partial overlap; finding first prominent features in a subregion G1 and a subregion G2 in a region of the partial overlap; determining first geodesics based on the first prominent features; finding second prominent features in a subregion H1 and a subregion H2 in the region of the partial overlap; determining second geodesics based on the second prominent features; generating a first correspondence between the subregion G1 and the subregion H1; generating a second correspondence between the subregion G2 and the subregion H2; deforming the first geodesics and the second geodesics based on the first correspondence and the second correspondence; stitching the first point cloud and the second point cloud, thereby generating a stitched point cloud; calculating a required deformation to transform the first geodesics to the second geodesics; applying the required deformation to the second point cloud to match the second point cloud to the first point cloud, thereby refining the stitched point cloud; and producing a refined stitched point cloud in a predetermined format. ### Claim 13 (depends on 12) The computer-implemented method of claim 12, wherein finding the first prominent features comprises: estimating a curvature in the subregion G1 or the subregion G2; then refining the estimated curvature by iteratively calculating the curvature over one or more iterations, such that a region of high curvature is re-calculated by considering a smaller neighborhood in the region of high curvature; calculating the curvature based on the smaller neighborhood; and repeating the iterations until sampling in a smaller neighborhood does not increase the curvature in the region of high curvature by more than a specified threshold. ### Claim 14 (depends on 12) The computer-implemented method of claim 12, wherein finding the second prominent features comprises: estimating a curvature in the subregion H1 or the subregion H2; then refining the estimated curvature by iteratively calculating the curvature over one or more iterations, such that a region of high curvature is re-calculated by considering the smaller neighborhood in the region of high curvature; calculating the curvature based on the smaller neighborhood; and repeating the iterations until sampling in the smaller neighborhood does not increase the curvature in the region of high curvature by more than a specified threshold. ### Claim 15 (depends on 12) The computer-implemented method of claim 12, wherein: finding the first prominent features comprises approximating a curvature on the subregion G1 and the subregion G2 by: discretizing a curvature operator; and extracting the first prominent features with respect to the curvature where the curvature is high or zero, a high curvature representing sharp corners and a zero curvature representing a flat region. ### Claim 16 (depends on 12) The computer-implemented method of claim 12, wherein finding the second prominent features comprises: approximating a curvature on the subregion H1 and the subregion H2 by: discretizing a curvature operator; and extracting the second prominent features with respect to the curvature where the curvature is high or zero, a high curvature representing sharp corners and a zero curvature representing a flat region. ### Claim 17 (depends on 12) The computer-implemented method of claim 12, wherein computing the first geodesics and computing the second geodesics comprise: triangulating on respective subregions; and determining a shortest path between two points on a triangulated surface. ### Claim 18 (depends on 12) The computer-implemented method of claim 12, wherein generating the first correspondence and generating the second correspondence comprise: ranking the first prominent features and the second prominent features, respectively, by curvature in the first point cloud and the second point cloud, respectively, and matching points between the first point cloud and the second point cloud. ### Claim 19 (depends on 12) The computer-implemented method of claim 12, wherein calculating the required deformation comprises: producing a first surface determined by connecting any two points between the subregion G1 and the subregion H1; producing a geodesic, and extending the geodesic into a second surface by considering a neighborhood about the geodesic. ### Claim 20 (depends on 12) The computer-implemented method of claim 12, wherein calculating the required deformation further comprises: estimating a first fundamental form of a surface by finite difference methods; and deforming the surface by an As Rigid As Possible (ARAP) deformation using the first correspondence and the second correspondence. ### Claim 21 (depends on 20) The computer-implemented method of claim 20, wherein calculating the required deformation further comprises: determining a similarity between deformed subregions by: estimating a second fundamental form for each of the subregions; and comparing a coefficient between the first fundamental form and the second fundamental form. ### Claim 22 (depends on 12) The computer-implemented method of claim 12, wherein measurements from the refined stitched point cloud, are extracted with an employment of a geodesic distance function. ### Claim 23 A non-transitory computer-readable medium comprising a plurality of instructions that are executable on a processor of a system to stitch deformable objects, the instructions comprising instructions to: receive a first point cloud and a second point cloud having a partial overlap; find first prominent features in a subregion G1 and a subregion G2 in a region of the partial overlap; determine first geodesics based on the first prominent features; find second prominent features in a subregion H1 and a subregion H2 in the region of the partial overlap; determine second geodesics based on the second prominent features; generate a first correspondence between the subregion G1 and the subregion H1; generate a second correspondence between the subregion G2 and the subregion H2; deform the first geodesics and the second geodesics based on the first correspondence and the second correspondence; stitch the first point cloud and the second point cloud, thereby generating a stitched point cloud; calculate a required deformation to transform the first geodesics to the second geodesics; apply the required defamation deformation to the second point cloud to match the second point cloud to the first point cloud, thereby refining the stitched point cloud; and produce a refined stitched point cloud in a predetermined format. ### Claim 24 (depends on 23) The non-transitory computer-readable medium of claim 23, wherein the instructions comprise instructions to: extract measurements from the refined stitched point cloud with an employment of a geodesic distance function.