# US 12,222,422 B2 · Post-processing of mapping data for improved accuracy and noise-reduction

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,222,422 B2
- Document ID: US12222422B2
- Status: Granted
- Kind: B2
- Application number: 17/997,997
- Filed: 2021-06-22
- Published: 2023-06-08
- Granted: 2025-02-11
- Earliest claimed: 2020-06-26
- Assignees: DJI Research LLC
- Applicants: DJI Research LLC
- Inventors: Zhenyu Yang; Zhenghe Shangguan; Arjun Sukumar Menon; Weifeng Liu
- Web page: https://patentgrep.com/patent/US12222422B2

## CPC

- G01S 17/89: for mapping or imaging
- G01S 7/4808: {Evaluating distance, position or velocity data}
- G01S 17/42: Simultaneous measurement of distance and other co-ordinates (indirect measurement G01S17/46)
- G01S 17/86: Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
- G01S 17/88: Lidar systems specially adapted for specific applications
- G06T 7/337: {involving reference images or patches}
- G06T 11/10: Texturing; Colouring; Generation of textures or colours (retouching, inpainting or scratch removal G06T5/77)
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Techniques are disclosed for post-processing mapping data in a movable object environment. A method for post-processing mapping data can include obtaining a plurality of scans, each scan comprising georeferenced mapping data and corresponding payload pose data obtained from a payload supported by an unmanned aerial vehicle (UAV), generating a plurality of local maps based on matching portions of the plurality of scans using the georeferenced mapping data, generating a pose graph based on identifying a plurality of correspondence points among the plurality of local maps, and optimizing the pose graph by minimizing an error distance between each pair of correspondence points to obtain optimized transforms for two scans among the plurality of the scans.

## Claims

### Claim 1

A method, comprising:
obtaining a plurality of scans, each scan comprising georeferenced mapping data and corresponding payload pose data obtained from a payload supported by a movable object;
generating a plurality of local maps based on matching portions of the plurality of scans using the georeferenced mapping data;
generating a pose graph from the plurality of local maps, including identifying a plurality of pairs of correspondence points each including two correspondence points from two of the plurality of local maps, respectively, and representing a same point in space; and
optimizing the pose graph by minimizing a distance between the two correspondence points in each of the plurality of pairs of correspondence points to obtain optimized transforms each for two of the plurality of the scans.

### Claim 2 (depends on 1)

The method of claim 1, wherein the plurality of scans includes a plurality of overlapping scans of a target region or a target object.

### Claim 3 (depends on 2)

The method of claim 2, wherein generating the plurality of local maps based on matching portions of the plurality of scans using the georeferenced mapping data, further comprises:
identifying at least one pair of overlapping scans from the plurality of scans based on point cloud data associated with the at least one pair of overlapping scans; and
combining the at least one pair of overlapping scans into a local map.

### Claim 4 (depends on 3)

The method of claim 3, wherein identifying the at least one pair of overlapping scans from the plurality of scans based on the point cloud data associated with the at least one pair of overlapping scans, further comprises:
determining the at least one pair of overlapping scans are overlapping using iterative closest point (ICP) matching.

### Claim 5 (depends on 1)

The method of claim 1, wherein identifying the plurality of pairs of correspondence points, further comprises:
determining the plurality of pairs of correspondence points using iterative closest point (ICP) matching.

### Claim 6 (depends on 5)

The method of claim 5, wherein determining the plurality of pairs of correspondence points using iterative closest point (ICP) matching, further comprises:
performing ICP matching directly on the georeferenced mapping data.

### Claim 7 (depends on 1)

The method of claim 1, wherein optimizing the pose graph by minimizing the distance between the two correspondence points in each of the plurality of pairs of correspondence points to obtain the optimized transforms, further comprises:
determining one or more cost functions, each cost function corresponding to a pair of correspondence points from the plurality of correspondence points; and
globally solving the one or more cost functions to obtain the optimized transforms.

### Claim 8 (depends on 1)

The method of claim 1, further comprising:
generating optimized point cloud data using the optimized transforms and the plurality of scans.

### Claim 9 (depends on 8)

The method of claim 8, further comprising:
obtaining color data from an RGB camera; and
overlaying the color data on the optimized point cloud data.

### Claim 10 (depends on 1)

The method of claim 1, wherein the payload includes a LiDAR sensor and an inertial measurement unit.

### Claim 11

A system, comprising:
a movable object;
a payload coupled to the movable object;
a computing device including at least one processor and a memory, the memory including instructions which, when executed by the at least one processor, cause the computing device to:
obtain a plurality of scans from the payload, each scan comprising georeferenced mapping data and corresponding payload pose data obtained from the payload;
generate a plurality of local maps based on matching portions of the plurality of scans using the georeferenced mapping data;
generate a pose graph from the plurality of local maps, including identifying a plurality of pairs of correspondence points each including two correspondence points from two of the plurality of local maps, respectively, and representing a same point in space; and
optimize the pose graph by minimizing a distance between the two correspondence points in each of the plurality of pairs of correspondence points to obtain optimized transforms each for two of the plurality of the scans.

### Claim 12 (depends on 11)

The system of claim 11, wherein the plurality of scans includes a plurality of overlapping scans of a target region or a target object.

### Claim 13 (depends on 12)

The system of claim 12, wherein to generate the plurality of local maps based on matching portions of the plurality of scans using the georeferenced mapping data, the instructions, when executed, further cause the computing device to:
identify at least one pair of overlapping scans from the plurality of scans based on point cloud data associated with the at least one pair of overlapping scans; and
combine the at least one pair of overlapping scans into a local map.

### Claim 14 (depends on 13)

The system of claim 13, wherein to identify the at least one pair of overlapping scans from the plurality of scans based on the point cloud data associated with the at least one pair of overlapping scans, the instructions, when executed, further cause the computing device to:
determine the at least one pair of overlapping scans are overlapping using iterative closest point (ICP) matching.

### Claim 15 (depends on 13)

The system of claim 13, wherein to optimize the pose graph by minimizing the distance between the two correspondence points in each of the plurality of pairs of correspondence points to obtain the optimized transforms, the instructions, when executed, further cause the computing device to:
determine a plurality of one or more cost functions, each cost function corresponding to a pair of correspondence points from the plurality of correspondence points; and
globally solve the plurality of one or more cost functions to obtain the optimized transforms.

### Claim 16 (depends on 13)

The system of claim 13, wherein the instructions, when executed, further cause the computing device to:
generate optimized point cloud data using the optimized transforms and the plurality of scans.

### Claim 17 (depends on 11)

The system of claim 11, wherein to identify the plurality of pairs of correspondence points, the instructions, when executed, further cause the computing device to:
determine the plurality of pairs of correspondence points using iterative closest point (ICP) matching.

### Claim 18 (depends on 17)

The system of claim 17, wherein to determine the plurality of pairs of correspondence points using iterative closest point (ICP) matching, the instructions, when executed, further cause the computing device to:
perform ICP matching directly on the georeferenced mapping data.

### Claim 19

A non-transitory computer readable storage medium including instructions stored thereon which, when executed by a processor, cause the processor to:
obtain a plurality of scans, each scan comprising georeferenced mapping data and corresponding payload pose data obtained from a payload supported by a movable object;
generate a plurality of local maps based on matching portions of the plurality of scans using the georeferenced mapping data;
generate a pose graph from the plurality of local maps, including identifying a plurality of pairs of correspondence points each including two correspondence points from two of the plurality of local maps, respectively, and representing a same point in space; and
optimize the pose graph by minimizing a distance between the two correspondence points in each of the plurality of pairs of correspondence points to obtain optimized transforms each for two of the plurality of the scans.

### Claim 20 (depends on 19)

The non-transitory computer readable storage medium of claim 19, wherein the plurality of scans includes a plurality of overlapping scans of a target region.