# US 2023/0274450 A1 · Pavement macrotexture determination using multi-view smartphone images

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0274450 A1
- Document ID: US20230274450A1
- Status: Published application
- Kind: A1
- Application number: 18/143,524
- Filed: 2023-05-04
- Published: 2023-08-31
- Earliest claimed: 2020-03-14
- Assignees: Purdue Research Foundation
- Applicants: Purdue Research Foundation
- Inventors: Jie Shan; Xiangxi Tian
- Web page: https://patentgrep.com/patent/US20230274450A1

## CPC

- G06T 7/41: based on statistical description of texture
- G06T 7/593: from stereo images
- G06T 7/596: {from three or more stereo images}
- G06V 30/18143: {Extracting features based on salient regional features, e.g. scale invariant feature transform [SIFT] keypoints}
- G06T 2207/10021: Stereoscopic video; Stereoscopic image sequence
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/20021: Dividing image into blocks, subimages or windows
- G06T 2207/30132: Masonry; Concrete
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A method of determining macrotexture of an object is disclosed which includes obtaining a plurality of stereo images from an object by an imaging device, generating a coordinate system for each image of the plurality of stereo images, detecting one or more keypoints each having a coordinate in each image of the plurality of stereo images, wherein the coordinate system is based on a plurality of ground control points (GCPs) with apriori position knowledge of each of the plurality of GCPs, generating a sparse point cloud based on the one or more keypoints, reconstructing a 3D dense point cloud of the object based on the generated sparse point cloud and based on neighboring pixels of each of the one or more keypoints and calculating the coordinates of each pixel of the 3D dense point cloud, and obtaining the macrotexture based on the reconstructed 3D dense point cloud of the object.

## Claims

### Claim 1

A method of determining macrotexture of an object, comprising:
obtaining a plurality of stereo images from an object collected from a plurality of angles by an imaging device;
generating a coordinate system for each image of the plurality of stereo images;
detecting one or more keypoints each having a coordinate in each image of the plurality of stereo images, wherein the coordinate system is based on a plurality of ground control points (GCPs) with apriori position knowledge of each of the plurality of GCPs;
generating a sparse point cloud based on the one or more keypoints;
reconstructing a 3D dense point cloud of the object based on the generated sparse point cloud and based on neighboring pixels of each of the one or more keypoints and calculating the coordinates of each pixel of the 3D dense point cloud;
obtaining the macrotexture based on the reconstructed 3D dense point cloud of the object, comprising:
dividing the 3D dense point cloud into a plurality of segments based on a first predetermined distance criterion;
subdividing each divided segment of the plurality of segments into a plurality of subdivided segments based on a second predetermined distance criterion;
determining peaks and valleys for each subdivided segment of the plurality of segments;
determining the maximum peak in each subdivided segment of the plurality of subdivided segments of each segment of the plurality of segments;
obtaining a mean segment depth by averaging the maximum peaks for each subdivided segment of the plurality of subdivided segments of each segment of the plurality of segments; and
obtaining a mean profile depth of the object by averaging the mean segment depths for each segment of the plurality of segments.

### Claim 2 (depends on 1)

The method of claim 1, wherein each two consecutive images of the plurality of stereo images have an overlap of between about 70% to about 100%.

### Claim 3 (depends on 2)

The method of claim 2, wherein the neighboring pixels of each of the one or more keypoints are selected based on a predetermined number of pixel surrounding the one or more keypoints.

### Claim 4 (depends on 3)

The method of claim 3, wherein the coordinates of each of the neighboring pixels are averaged and assigned a value for an associated point in the 3D dense point cloud.

### Claim 5 (depends on 1)

The method of claim 1, wherein the object is a pavement.

### Claim 6 (depends on 1)

The method of claim 1, wherein the imaging device is a smartphone.

### Claim 7 (depends on 1)

The method of claim 1, wherein the step of detecting the one or more keypoints in the plurality of images is by i) identifying a pixel with disparate pixel characteristics as compared to neighboring pixels according to a predetermined threshold, and ii) matching pixel attributes between the one or more keypoints on each image of the plurality of stereo images.

### Claim 8 (depends on 1)

The method of claim 1, wherein the step of generating the sparse point cloud includes estimating interior and exterior orientation parameters for each camera position of the plurality of angles based on relative position and orientation of each camera of the plurality of cameras to the object.

### Claim 9 (depends on 1)

The method of claim 1, wherein the first distance criterion is based on dividing the 3D dense point cloud into a predetermined equally divided segments.

### Claim 10 (depends on 9)

The method of claim 9, wherein the predetermined equally divided segments is about 10.

### Claim 11 (depends on 1)

The method of claim 1, wherein the second distance criterion is based on dividing each segment into an equally long subdivided segment.

### Claim 12

A method of determining macrotexture of an object, comprising:
obtaining a plurality of stereo images from an object collected from a plurality of angles by an imaging device;
generating a coordinate system for each image of the plurality of stereo images;
detecting one or more keypoints each having a coordinate in each image of the plurality of stereo images, wherein the coordinate system is based on a plurality of ground control points (GCPs) with apriori position knowledge of each of the plurality of GCPs;
generating a sparse point cloud based on the one or more keypoints;
reconstructing a 3D dense point cloud of the object based on the generated sparse point cloud and based on neighboring pixels of each of the one or more keypoints and calculating the coordinates of each pixel of the 3D dense point cloud; and
obtaining the macrotexture based on the reconstructed 3D dense point cloud of the object;
wherein each two consecutive images of the plurality of stereo images have an overlap of between about 70% to about 100%,
wherein the neighboring pixels of each of the one or more keypoints are selected based on a predetermined number of pixel surrounding the one or more keypoints, and
wherein the coordinates of each of the neighboring pixels are averaged and assigned a value for an associated point in the 3D dense point cloud.

### Claim 13 (depends on 12)

The method of claim 12, further comprising:
dividing the 3D dense point cloud into a plurality of segments based on a first predetermined distance criterion;
subdividing each divided segment of the plurality of segments into a plurality of subdivided segments based on a second predetermined distance criterion;
obtaining an average of peaks and valleys for each segment of the plurality of segments;
determining the maximum peak in each subdivided segment of the plurality of subdivided segments of each segment of the plurality of segments;
obtaining a mean segment depth by averaging the maximum peaks for each subdivided segment of the plurality of subdivided segments of each segment of the plurality of segments; and
obtaining a mean profile depth of the object by averaging the mean segment depths for each segment of the plurality of segments.

### Claim 14 (depends on 12)

The method of claim 12, wherein the object is a pavement.

### Claim 15 (depends on 12)

The method of claim 12, wherein the imaging device is a smartphone.

### Claim 16 (depends on 12)

The method of claim 12, wherein the step of detecting the one or more keypoints in the plurality of images is by i) identifying a pixel with disparate pixel characteristics as compared to neighboring pixels according to a predetermined threshold, and ii) matching pixel attributes between the one or more keypoints on each image of the plurality of stereo images.

### Claim 17 (depends on 12)

The method of claim 12, wherein the step of generating the sparse point cloud includes estimating interior and exterior orientation parameters for each camera position of the plurality of angles based on relative position and orientation of each camera of the plurality of cameras to the object.

### Claim 18 (depends on 13)

The method of claim 13, wherein the first distance criterion is based on dividing the 3D dense point cloud into a predetermined equally divided segments.

### Claim 19 (depends on 18)

The method of claim 18, wherein the predetermined equally divided segments is about 10.

### Claim 20 (depends on 13)

The method of claim 13, wherein the second distance criterion is based on dividing each segment into an equally long subdivided segment.
