# US 2013/0155061 A1 · Autonomous pavement condition assessment > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2013/0155061 A1 - Document ID: US20130155061A1 - Status: Published application - Kind: A1 - Application number: 13/717,244 - Filed: 2012-12-17 - Published: 2013-06-20 - Earliest claimed: 2011-12-16 - Assignees: University of Southern California - Inventors: Mohammad R. Jahanshahi; Farrokh Jazizadeh Karimi; Sami F. Masri; Burcin Becerik-Gerber - Web page: https://patentgrep.com/patent/US20130155061A1 ## CPC - G01C 7/02: of land surfaces - G06T 7/0004: {Industrial image inspection} - G06T 7/11: Region-based segmentation - G06T 7/136: involving thresholding - G06T 15/00: Three-dimensional [3D] image rendering - G06T 2207/10028: Range image; Depth image; 3D point clouds - G06T 2207/30132: Masonry; Concrete - G06T 2207/30256: Lane; Road marking - G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract An autonomous pavement assessment system may receive depth data indicative of the depth of pixels that collectively comprise multiple defective areas of pavement. For each defective area, the system may fit a plane to it; generate a histogram that indicates the frequency of its pixels at different depths; dynamically determine a depth noise threshold for it; generate a binary image of it based on its noise threshold; and generate a depth map of it containing only the pixels that have a depth that meets or exceeds its depth noise threshold. The system may prioritize the multiple defective areas for repair and/or generate optimized rehabilitation routes. Crowd sourcing may be used to gather the depth data, as well as location information for each defective area. ## Claims ### Claim 1 A non-transitory, tangible, computer-readable storage media containing a program of instructions configured to cause a computer system running the program of instructions to: receive depth data indicative of the depth of pixels that collectively comprise a defective area of pavement; dynamically determine a depth noise threshold based on characteristics of the depth data; and generate a depth map of only the pixels that comprise the defective area that have a depth that meets or exceeds the depth noise threshold. ### Claim 2 (depends on 1) The media of claim 1 wherein the program of instructions is configured to cause the computer system running the program of instructions to: generate a binary image of the defective area based on the noise threshold, the binary image delineating pixels that meet or exceed the noise threshold as a central portion of the defective area and pixels that fail to exceed the noise threshold as a surrounding portion of the defective area; and restrict the area covered by the depth map to the central portion of the defective area. ### Claim 3 (depends on 1) The media of claim 1 wherein the program of instructions is configured to cause the computer system running the program of instructions to: generate a histogram of the defective area that indicates the frequency of pixels at different depths within the defective area; and determine the depth noise threshold based on the histogram. ### Claim 4 (depends on 3) The media of claim 3 wherein the program of instructions is configured to cause the computer system running the program of instructions to determine the depth noise threshold using the Otsu method. ### Claim 5 (depends on 1) The media of claim 1 wherein the program of instructions is configured to cause the computer system running the program of instructions to fit a plane to the defective area that passes through an upper surface of the defective area. ### Claim 6 (depends on 5) The media of claim 5 wherein the program of instructions is configured to cause the computer system running the program of instructions to fit the plane using RANSAC. ### Claim 7 (depends on 5) The media of claim 5 wherein the program of instructions is configured to cause the computer system running the program of instructions to subtract the depth data for the defective area from the fitted plane. ### Claim 8 (depends on 1) The media of claim 1 wherein the program of instructions is configured to cause the computer system running the program of instructions to determine the surface area of the defective area demarcated by the depth map. ### Claim 9 (depends on 8) The media of claim 8 wherein the program of instructions is configured to cause the computer system running the program of instructions to determine the volume of the defective area demarcated by the depth map. ### Claim 10 (depends on 9) The media of claim 9 wherein the program of instructions is configured to cause the computer system running the program of instructions to determine the depth of the defective area demarcated by the depth map. ### Claim 11 (depends on 10) The media of claim 10 wherein the program of instructions is configured to cause the computer system running the program of instructions to determine the length and width of the defective area demarcated by the depth map. ### Claim 12 (depends on 1) The media of claim 1 wherein the program of instructions is configured to cause the computer system running the program of instructions to: receive depth data indicative of the depth of pixels that collectively comprise multiple defective areas of pavement; identify the multiple defective areas of pavement based on the depth data; receive and store location information indicative of the location of each defective area; determine a depth noise threshold for each defective area based on characteristics of the depth data for that defective area; and generate a depth map for each defective area that includes only the pixels that comprise the defective area that meet or exceed the depth noise threshold for that defective area. ### Claim 13 (depends on 12) The media of claim 12 wherein the program of instructions is configured to cause the computer system running the program of instructions to prioritize the multiple defective areas for repair based on the depth maps. ### Claim 14 (depends on 12) The media of claim 12 wherein the program of instructions is configured to cause the computer system running the program of instructions to generate optimized rehabilitation routes based on the depth maps and the location information. ### Claim 15 (depends on 1) The media of claim 1 wherein: the depth data is indicative of the depth of the pixels at multiple, different times; and the program of instructions is configured to cause the computer system running the program of instructions to project the likely progress of the defect area based on the depth data. ### Claim 16 (depends on 1) The media of claim 1 wherein the program of instructions is configured to cause the computer system running the program of instructions to receive an RGB image of the defective area of pavement and to utilize this RGB image in an analysis of the defective area. ### Claim 17 (depends on 1) The media of claim 1 wherein the program of instructions to generate and analyze the depth map in an unsupervised manner without training of the computer system. ### Claim 18 A non-transitory, tangible, computer-readable storage media containing a program of instructions configured to cause the computer system running the program of instructions to: receive depth data indicative of the depth of pixels that collectively comprise multiple defective areas of pavement; identify the multiple defective areas of pavement based on the depth data; receive and store information indicative of the location of each defective area; categorize each defective area based on the depth data; and prioritize the multiple defective areas for repair based on the categorization, whereby the identifying, categorizing, and prioritizing proceed in an automated and unsupervised manner without training of the computer system. ### Claim 19 An autonomous pavement condition assessment system comprising: multiple vehicles, each equipped with: a depth sensor configured to generate depth data indicative of the depth of pixels that collectively comprise pavement over which the vehicles travel; and a location sensor configured to generate location data indicative of the locations of the pavement at which the depth data is generated; a computer system, including computer hardware, configured to: receive the depth data and the location data; to identify and classify defective areas of the pavement based on the depth data; and to prioritize the multiple defective areas for repair based on the identification, the classification, and the location data. ### Claim 20 (depends on 19) The autonomous pavement condition assessment system of claim 19 wherein: the depth data includes data about the same defective area in the pavement from more than one of the vehicles; and the computer system is configured to eliminate redundancies in the depth data caused by the data about the same defective area from the more than one of the vehicles. ### Claim 21 (depends on 19) The autonomous pavement condition assessment system of claim 19 wherein the depth sensor is Microsoft Kinect.