# US 11,908,150 B2 · System and method for mobile 3D scanning and measurement > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,908,150 B2 - Document ID: US11908150B2 - Status: Granted - Kind: B2 - Application number: 18/187,089 - Filed: 2023-03-21 - Published: 2023-07-20 - Granted: 2024-02-20 - Earliest claimed: 2019-11-28 - Assignees: Xesto Inc. - Applicants: Xesto Inc. - Inventors: Mehmet Afiny Affan Akdemir; Christian Garcia Salguero; Victoria Sophie Howe - Web page: https://patentgrep.com/patent/US11908150B2 ## CPC - G06T 7/593: from stereo images - G06T 17/00: Three-dimensional [3D] modelling for computer graphics - G06T 7/33: using feature-based methods - G06T 7/579: from motion - G06T 19/00: Manipulating three-dimensional [3D] models or images for computer graphics - G06T 2207/10028: Range image; Depth image; 3D point clouds - G06T 2207/20036: Morphological image processing - G06T 2207/20081: Training; Learning - G06T 2207/20084: Artificial neural networks [ANN] - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract Systems and methods are provided for three-dimensional scanning and measurement by a device having a processor. The processor is configured to receive images of an object from at least two angles; preprocess the images using morphological refinement; create a source point cloud based on the images; remove outliers from the source point cloud; globally register the source point cloud to generate a transformed source point cloud; compare the transformed source point cloud with a target point cloud to generate a stitched point cloud that thereby creates a stitched 3D model; measure the resulting stitched 3D model; and provide the resulting stitched 3D model for comparison to a digitized item to assess sizing of the 3D model to the item. ## Claims ### Claim 1 A system of three-dimensional scanning and measurement, comprising: a mobile device to scan a plurality of images of an object; a trained model of a neural network configured by parameter optimization; at least one processor coupled to the mobile device to process the plurality of images of the object; and a non-transitory computer-readable medium having stored thereon computer-executable instructions that, when executed by the at least one processor, cause the system to: receive the plurality of images of the object, preprocess the plurality of images, employing morphological refinement; create a plurality of point clouds based on the plurality of images; determine, for each point cloud, a plurality of prominent features by a determination of: an estimate of a curvature of each of the plurality of point clouds; a refinement of the estimate of the curvature by an iterative calculation of the curvature over one or more iterations such that a region of high curvature is calculated by considering a smaller neighborhood in the region; and an iterative repetition until a sample in the smaller neighborhood does not increase the curvature in the region by more than a specified threshold; rank the prominent features within in each point cloud by the curvature determined therefor; match points between the point clouds based on a similarity of the curvature determined therefor; generate a transformed point cloud based on the prominent features in each point cloud of the plurality of point clouds; and determine an overlap between the plurality of point clouds and labeled correspondences between prominent features thereof. ### Claim 2 (depends on 1) The system of claim 1, wherein the computer-readable medium comprises instruction to: transform the point clouds based on the match between the labeled correspondences therebetween. ### Claim 3 (depends on 1) The system of claim 1, wherein the morphological refinement comprises: an employment of a two-dimensional (2D) or three-dimensional (3D) model, to generate morphological changes by an erosion of points in a 3D model consisting of a 3D point cloud. ### Claim 4 (depends on 1) The system of claim 1, wherein the morphological refinement comprises: an employment of a two-dimensional (2D) or three-dimensional (3D) model, to generate morphological changes via edge refinement. ### Claim 5 (depends on 1) The system of claim 1, wherein the computer-readable medium comprises instruction to: ingest the transformed point cloud by the trained model, the transformed point cloud having a surface with a discontinuity on the surface and a first radius for each point around the discontinuity, wherein the neural network is configured to generate a second radius for each point around the discontinuity that corresponds to regions to be removed from the transformed point cloud. ### Claim 6 (depends on 1) The system of claim 1, wherein the instructions to determine the prominent feature comprise instructions to: accentuate regions of the point cloud with high curvature using at least one of 2D or 3D inputs. ### Claim 7 (depends on 1) The system of claim 1, wherein the computer-readable medium comprises instruction to: measure the transformed point cloud by an employment of a geodesic distance function. ### Claim 8 (depends on 7) The system of claim 7, wherein the computer-readable medium comprises instructions to compute and compare the measurement of two of the transformed point clouds to one another. ### Claim 9 A computer-implemented method of three-dimensional scanning and measurement, comprising: scanning, by a mobile device, a plurality of images of an object; receiving, by a processor, the plurality of images; preprocessing, by the processor, each of the plurality of images using morphological refinement; creating, by the processor, a plurality of point clouds from the preprocessed plurality of images; determining, by the processor and for each point cloud, prominent features by: estimating, by the processor, a curvature of each of the plurality of point clouds; and thereafter refining, by the processor, the estimated curvature by iteratively calculating the curvature over one or more iterations, such that a region of high curvature is re-calculated by a consideration of a smaller neighborhood in the region; and repeating, by the processor, the iterations until sampling in the smaller neighborhood does not increase the curvature in the region by more than a specified threshold; ranking, by the processor, the prominent features in each point cloud the based on the curvature determined therefor, matching, by the processor, points with similar curvature based on the curvature determined therefor; generating, by the processor, a transformed point cloud based on the prominent features in each point cloud of the plurality of point clouds; and determining, by a trained model, an overlap between the plurality of point clouds and labeled correspondences between prominent features thereof. ### Claim 10 (depends on 9) The method of claim 9, comprising: transforming the plurality of point clouds based on matching the labeled correspondences therebetween. ### Claim 11 (depends on 9) The method of claim 9, wherein using morphological refinement comprises: employing a two-dimensional (2D) or three-dimensional (3D) model to generate morphological changes by an erosion of points in a 3D model consisting of a 3D point cloud. ### Claim 12 (depends on 9) The method of claim 9, wherein using morphological refinement comprises: employing a two-dimensional (2D) or three-dimensional (3D) model to generate morphological changes via edge refinement. ### Claim 13 (depends on 9) The method of claim 9, further comprising: ingesting, by the trained model, the transformed point cloud having a surface with a discontinuity on the surface and a first radius for each point around the discontinuity; and generating, by the trained model, a second radius for each point around the discontinuity that corresponds to regions to be removed from the transformed point cloud. ### Claim 14 (depends on 9) The method of claim 9, furthermore comprising: accentuating, by the processor, regions of the point cloud with high curvature using either 2D or 3D inputs. ### Claim 15 (depends on 9) The computer-implemented method of claim 9, comprising: measuring the transformed point cloud by a geodesic distance function. ### Claim 16 (depends on 15) The computer-implemented method of claim 15, wherein the measurement of two of the transformed point clouds are computed and compared to one another. ### Claim 17 (depends on 9) The computer-implemented method of claim 9, wherein the object is a foot. ### Claim 18 A non-transitory computer-readable medium having stored thereon computer- executable instructions that, when executed by a system to scan and measure three-dimensional features, the instructions comprising instructions to: receive a plurality of images of an object; preprocess each of the plurality of images using morphological refinement; create a plurality of point clouds from the preprocessed plurality of images; determine, for each point cloud, prominent features by a determination of: an estimate of a curvature of each of the plurality of point clouds; and thereafter a refinement of the estimated curvature by an iterative calculation of the curvature over one or more iterations, such that a region of high curvature is re-calculated by a consideration of a smaller neighborhood in the region; and a repetition of the iterations until a sample of the smaller neighborhood does not increase the curvature in the region by more than a specified threshold; rank the prominent features in each point cloud the based on the curvature determined therefor; match points with similar curvature based on the curvature determined therefor; generate a transformed point cloud based on the prominent features in each point cloud of the plurality of point clouds; and determine an overlap between the plurality of point clouds and labeled correspondences between prominent features thereof. ### Claim 19 (depends on 18) The non-transitory computer-readable medium of claim 18, wherein the morphological refinement is performed by 3D preprocessing to cause morphological changes by at least one of: an erosion of points in a 3D model consisting of a 3D point cloud; or edge refinement. ### Claim 20 (depends on 18) The non-transitory computer-readable medium of claim 18, wherein the computer readable instructions include instructions to: accentuate regions of the point cloud with high curvature using either 2D or 3D inputs. ### Claim 21 (depends on 18) The non-transitory computer-readable medium of claim 18, wherein the computer readable instructions include instructions to: measure the transformed point cloud by a geodesic distance function. ### Claim 22 (depends on 21) The non-transitory computer-readable medium of claim 21, wherein the computer readable instructions include instructions to: compute and compare the measurement of two of the transformed point clouds to one another. ### Claim 23 (depends on 18) The non-transitory computer-readable medium of claim 18, wherein the computer readable instructions include instructions to: ingest, by a trained model of the system, the transformed point cloud having a surface with a discontinuity on the surface and a first radius for each point around the discontinuity; and generate, by the trained model, a second radius for each point around the discontinuity that corresponds to regions to be removed from the transformed point cloud.