# US 12,002,156 B2 · Point cloud filtering > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,002,156 B2 - Document ID: US12002156B2 - Status: Granted - Kind: B2 - Application number: 18/339,424 - Filed: 2023-06-22 - Published: 2023-10-26 - Granted: 2024-06-04 - Earliest claimed: 2021-04-14 - Assignees: Lineage Logistics, LLC - Applicants: Lineage Logistics, LLC - Inventors: Christopher Frank Eckman; Brady Michael Lowe; Alexander Hall - Web page: https://patentgrep.com/patent/US12002156B2 ## CPC - G01C 15/00: Surveying instruments or accessories not provided for in groups G01C1/00 - G01C13/00 - G06T 17/05: Geographic models - G06F 18/23: Clustering techniques - G06F 18/23213: with fixed number of clusters, e.g. K-means clustering - G06F 18/24137: {Distances to cluster centroïds} - G06T 17/20: Finite element generation, e.g. wire-frame surface description, {tesselation} - G06V 10/507: {Summing image-intensity values; Histogram projection analysis} - G06V 10/763: {Non-hierarchical techniques, e.g. based on statistics of modelling distributions} - G06V 20/10: Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59) - G06V 20/176: {Urban or other man-made structures} - G06V 20/182: {Network patterns, e.g. roads or rivers} - G06V 20/188: {Vegetation} - G01S 17/89: for mapping or imaging - G06T 2210/56: Particle system, point based geometry or rendering - G06V 2201/12: Acquisition of 3D measurements of objects - G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES ## Abstract This specification describes systems and methods for refining point cloud data. Methods can include receiving point cloud data for a physical space, iteratively selecting points along an x, y, and z dimension, clustering the selected points into 2D histograms, determining a slope value for each 2D histogram, and removing, based on the slope value exceeding a predetermined value, points from the point cloud data. Methods can also include iteratively voxelizing each 2D histogram into predetermined mesh sizes, summating points in each voxelized 2D histogram, removing, based on determining the summation is below a predetermined sum value, points from the point cloud data, keeping, based on determining that a number of points in each voxelized 2D histogram exceeds a threshold value, a center point, selecting, for each histogram, a point, identifying, nearest neighbors in the point cloud data, removing the identified nearest neighbors from the data, and returning remaining points. ## Claims ### Claim 1 A method for determining facility information using point cloud data, the method comprising: receiving, by a computing system, image data of a facility, wherein the image data is captured by at least one imaging device positioned within the facility; generating, by the computing system, point cloud data from the image data, wherein the point cloud data comprises a plurality of points in three-dimensional (“3D”) space that approximate locations of physical surfaces within the facility; filtering, by the computing system, the plurality of points in the point cloud data to (i) keep points in the point cloud data that correspond to contours of particular types of the physical surfaces within the facility and (ii) remove points from the point cloud data that do not correspond to the contours of the particular types of the physical surfaces within the facility, wherein the particular types of the physical surfaces comprise at least vertical poles and racks; generating, by the computing system, a map of the facility based at least in part on the points in the point cloud data that correspond to the contours of the particular types of the physical surfaces within the facility; determining, by the computing system, facility information based on the generated map, the facility information comprising at least identification of the vertical poles and the racks within the facility; and returning, by the computing system, the facility information. ### Claim 2 (depends on 1) The method of claim 1, wherein determining, by the computing system, facility information based on the generated map comprises determining a size or an occupancy of the facility that indicates an amount of physical space within the facility that includes the physical surfaces. ### Claim 3 (depends on 1) The method of claim 1, wherein the physical surfaces include racks, vertical poles, doorways, vehicles, human workers, and pallets. ### Claim 4 (depends on 1) The method of claim 1, wherein removing points from the point cloud data that do not correspond to the contours of the particular types of the physical surfaces within the facility comprises removing points from the point cloud data that correspond to walls and ceilings in the facility. ### Claim 5 (depends on 1) The method of claim 1, wherein keeping points in the point cloud data that correspond to the contours of the particular types of the physical surfaces within the facility comprises keeping points in the point cloud data that correspond to the racks and the vertical poles within the facility. ### Claim 6 (depends on 1) The method of claim 1, further comprising aligning, by the computing system, the filtered plurality of points in the point cloud data along x, y, and z dimensions that correspond to an orientation of the facility that is represented by the image data. ### Claim 7 (depends on 1) The method of claim 1, further comprising aligning, by the computing system, the point cloud data with a width of a wall in the facility to determine an orientation of the point cloud data that corresponds to an orientation of the facility. ### Claim 8 (depends on 1) The method of claim 1, wherein the facility is a warehouse, and determining, by the computing system, facility information based on the generated map comprises determining a capacity of the warehouse based on remaining points in the point cloud data. ### Claim 9 (depends on 1) The method of claim 1, further comprising classifying, by the computing system, the filtered plurality of points in the point cloud data as corresponding to one or more of the particular types of the physical surfaces within the facility. ### Claim 10 (depends on 9) The method of claim 9, wherein classifying, by the computing system, the filtered plurality of points in the point cloud data comprises applying a spherical filter to the filtered plurality of points. ### Claim 11 (depends on 9) The method of claim 9, wherein classifying, by the computing system, the filtered plurality of points in the point cloud data comprises: determining a confidence score for each of the particular types of the physical surfaces identified within the facility, the confidence score indicating a likelihood that the identified physical surface is a vertical pole or a rack; and removing, from the point cloud data, one or more points corresponding to the identified physical surface based on the confidence score being less than a threshold confidence value. ### Claim 12 (depends on 1) The method of claim 1, wherein filtering, by the computing system, the plurality of points in the point cloud data comprises: generating at least one histogram based on the point cloud data; iteratively voxelizing the at least one histogram into predetermined mesh sizes; filtering the at least one voxelized histogram based on a summation of points in the at least one voxelized histogram; identifying a set of points amongst the plurality of points in the at least one filtered histogram that are nearest neighbors to a center point amongst the set of points; and removing the set of points from the plurality of points in the image data. ### Claim 13 (depends on 1) The method of claim 1, wherein the at least one imaging device is a stereoscopic camera. ### Claim 14 (depends on 1) The method of claim 1, wherein the at least one imaging device is a drone. ### Claim 15 (depends on 1) The method of claim 1, further comprising determining, by the computing system, a process flow of activity in the facility based on the generated map of the facility. ### Claim 16 (depends on 1) The method of claim 1, wherein determining, by the computing system, facility information comprises determining a layout of the facility based at least in part on the points in the point cloud data that correspond to the contours of the particular types of the physical surfaces within the facility. ### Claim 17 (depends on 1) The method of claim 1, wherein the image data comprises at least one 3D scan of a portion of the physical space within the facility, wherein generating, by the computing system, point cloud data from the image data comprises interpolating points that correspond to other portions of the physical space within the facility that are not captured in the at least one 3D scan. ### Claim 18 (depends on 1) The method of claim 1, further comprising aligning, by the computing system, the point cloud data with an orientation of the facility based on identifying reference points in the point cloud data that correspond to doorways, doorframes, and windows within the facility. ### Claim 19 A system for determining facility information using point cloud data, the system comprising: a storage facility comprising walls, ceilings, vertical poles, and racks, the storage facility being configured to store pallets in one or more locations defined by at least the vertical poles and the racks; at least one imaging device positioned within the storage facility, wherein the at least one imaging device is configured to capture image data of a physical space within the storage facility; and a computing system in network communication with the at least one imaging device, the computing system comprising one or more processors and memory to map the storage facility and determine facility information based on the image data captured by the at least one imaging device, wherein the one or more processors are configured to execute operations comprising: receiving, from the at least one imaging device, the image data of a facility; generating point cloud data from the image data, wherein the point cloud data comprises a plurality of points in three-dimensional (“3D”) space that approximate locations of physical surfaces within the storage facility; filtering the plurality of points in the point cloud data to (i) keep points in the point cloud data that correspond to contours of particular types of the physical surfaces within the storage facility and (ii) remove points from the point cloud data that do not correspond to the contours of the particular types of the physical surfaces within the storage facility; generating a map of the storage facility based at least in part on the points in the point cloud data that correspond to the contours of the particular types of the physical surfaces within the storage facility; determining, by the computing system, facility information based on the generated map, the facility information comprising a size of the storage facility; and returning, by the computing system, the facility information. ### Claim 20 (depends on 19) The system of claim 19, wherein the operations further comprise: classifying the filtered plurality of points in the point cloud data as corresponding to one or more of the particular types of the physical surfaces within the storage facility; determining a confidence score for each of the particular types of the physical surfaces classified within the storage facility, the confidence score indicating a likelihood that the identified physical surface is a vertical pole or a rack; and removing, from the point cloud data, one or more points corresponding to the identified physical surface based on the confidence score being less than a threshold confidence value.