# US 2021/0241462 A1 · System and method for extracting planar surface from depth image

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0241462 A1
- Document ID: US20210241462A1
- Status: Published application
- Kind: A1
- Application number: 17/219,555
- Filed: 2021-03-31
- Published: 2021-08-05
- Earliest claimed: 2018-10-11
- Assignees: Shanghaitech University
- Applicants: Shanghaitech University
- Inventors: Ziran XING; Zhiru SHI; Yi MA; Jingyi YU
- Web page: https://patentgrep.com/patent/US20210241462A1

## CPC

- G06T 7/11: Region-based segmentation
- G06T 7/12: Edge-based segmentation
- G06T 7/162: involving graph-based methods
- G06T 7/187: involving region growing; involving region merging; involving connected component labelling
- G06T 7/50: Depth or shape recovery
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/20021: Dividing image into blocks, subimages or windows
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

According to some embodiments, an imaging processing method for extracting a plurality of planar surfaces from a depth map includes computing a depth change indication map (DCI) from a depth map in accordance with a smoothness threshold. The imaging processing method further includes recursively extracting a plurality of planar region from the depth map, wherein the size of each planar region is dynamically adjusted according to the DCI. The imaging processing method further includes clustering the extracted planar regions into a plurality of groups in accordance with a distance function; and growing each group to generate pixel-wise segmentation results and inlier points statistics simultaneously.

## Claims

### Claim 1

A method of imaging processing for extracting a plurality of planar surfaces from a depth map, comprising:
computing a depth change indication map (DCI) from a depth map in accordance with a smoothness threshold;
recursively extracting a plurality of planar region from the depth map, wherein the size of each planar region is dynamically adjusted according to the DCI;
clustering the extracted planar regions into a plurality of groups in accordance with a distance function; and
growing each group to generate pixel-wise segmentation results and inlier points statistics simultaneously.

### Claim 2 (depends on 1)

The method of claim 1, wherein each planar region comprises a rectangular area in the depth map.

### Claim 3 (depends on 1)

The method of claim 1, further comprising computing a plane normal and a plane distance for each planar region.

### Claim 4 (depends on 1)

The method of claim 1, further comprising measuring the smoothness of each planar region using a summed-area table (SAT) on DCI, wherein a mean square error (MSE) of each planar region is less than an MSE threshold (TMSE), and a curvature of each planar region is less than a curvature threshold (Tcuv).

### Claim 5 (depends on 1)

The method of claim 1, wherein the distance function comprises a normal threshold (Tnorm) and a distance threshold (Tdist).

### Claim 6 (depends on 1)

The method of claim 1, wherein the plurality of groups comprises a distinct set.

### Claim 7 (depends on 1)

The method of claim 1, further comprising clustering the extracted planar regions into a plurality of groups using a designed auto balanced search tree (AST).

### Claim 8 (depends on 7)

The method of claim 7, wherein the AST comprises a plurality of branches constructed by a red-black tree.

### Claim 9 (depends on 1)

The method of claim 1, wherein the number of groups is minimum, and the total number of pixels to the plurality of groups is maximum, and the plane fitting error is less than a predefined threshold.

### Claim 10 (depends on 1)

The method of claim 1, wherein the depth image is obtained from a 3D depth sensor, and the method is performed in real-time.

### Claim 11

An imaging processing system for extracting a plurality of planar surfaces from a depth map, comprising:
one or more processors; and
a memory storing instructions that, when executed by the one or more processors, cause the system to perform the steps of:
computing a depth change indication map (DCI) from a depth map in accordance with a smoothness threshold;
recursively extracting a plurality of planar region from the depth map, wherein the size of each planar region is dynamically adjusted according to the DCI;
clustering the extracted planar regions into a plurality of groups in accordance with a distance function; and
growing each group to generate pixel-wise segmentation results and inlier points statistics simultaneously.

### Claim 12 (depends on 11)

The system of claim 11, wherein each planar region comprises a rectangular area in the depth map.

### Claim 13 (depends on 11)

The system of claim 11, wherein the steps performed by the system further comprises computing a plane normal and a plane distance for each planar region.

### Claim 14 (depends on 11)

The system of claim 11, wherein the steps performed by the system further comprises measuring the smoothness of each planar region using a summed area table (SAT) on DCI, wherein a mean square error (MSE) of each planar region is less than an MSE threshold (TMSE), and a curvature of each planar region is less than a curvature threshold (Tcuv).

### Claim 15 (depends on 11)

The system of claim 11, wherein the distance function comprises a normal threshold (Tnorm) and a distance threshold (Tdist).

### Claim 16 (depends on 11)

The system of claim 11, wherein the plurality of groups comprises a distinct set.

### Claim 17 (depends on 11)

The system of claim 11, wherein the steps performed by the system further comprises clustering the extracted planar regions into a plurality of groups using a designed auto balanced search tree (AST).

### Claim 18 (depends on 17)

The system of claim 17, wherein the AST comprises a plurality of branches constructed by a red-black tree.

### Claim 19 (depends on 11)

The system of claim 11, wherein the number of groups is minimum, and the total number of pixels to the plurality of groups is maximum, and the plane fitting error is less than a predefined threshold.

### Claim 20 (depends on 11)

The system of claim 11, wherein the depth image is obtained from a 3D depth sensor, and the method is performed in real-time.
