US 12,737,976 B2Granted

Method for constructing structural semantic map under underground weak-light and low-texture environment

AssigneeTongji University

InventorsBin He, Gang Li, Feng Li, Yulong Ding, Bin Cheng, Zhongpan Zhu, Zhipeng Wang

Life of this patent

Estimating the term…

Loading patent…

Abstract

A method for constructing a structural semantic map under an underground weak-light and low-texture environment is provided. The method includes: fusing traditional methods with a parameter line detection and verification model of structural semantics of a Transformer; and establishing a geometric primitive half-plane search method guided by the direction of structural information. The method also includes establishing a neighborhood greedy expansion algorithm based on a geometric primitive model; and optimizing geometric primitive poses and boundaries one by one. The method further includes fusing a point cloud map with structural information, and establishing a semantic map with geometric structure primitives.

Claims10

Dependent claims

A method for constructing a structural semantic map under an underground weak-light and low-texture environment, comprising:

S1, acquiring, by a sensor based on vision and three-dimensional depth perception, multi-modal perception information of a target scene, wherein the multi-modal perception information comprises a Red-Green-Blue (RGB) visual image and depth perception information, and the depth perception information comprises an ordered point cloud and a disordered point cloud;
S2, acquiring a regular parameterized line set of the visual image through an Ellipse and Line Segment Detector with Continuous validation (ELSDc), and embedding and encoding line segments and ellipse arcs in the regular parameterized line set to obtain an embedding vector with a length of 256 dimensions;
S3, performing visual feature extraction on the visual image through an image feature extraction module;
S4, inputting the embedding vector and visual features into a trained parameterized line discrimination network model to obtain a structural semantic line set;
S5, preprocessing the ordered point cloud and the disordered point cloud in the depth perception information, and associating two-dimensional structural lines in the structural semantic line set with pixels of the visual image by indexing structural semantic line segments of the ordered point cloud and the disordered point cloud and passed pixels, to obtain an association relationship between the two-dimensional structural lines and three-dimensional points;
S6, performing three-dimensional space fitting on the two-dimensional structural lines based on the association relationship between the two-dimensional structural lines and the three-dimensional points, to obtain three-dimensional structural lines;
S7, performing geometric primitive half-plane search based on the three-dimensional structural lines, and determining whether point clouds on both sides of each three-dimensional structural line satisfy a constraint of a regular surface, if so, performing geometric primitive neighborhood expansion according to a greedy algorithm, and determining a maximum regular area;
and
S8, fusing the determined maximum regular area with a point cloud set to construct structural map data.

6 more elements

An apparatus for constructing a structural semantic map under an underground weak-light and low-texture environment, comprising:

a sensor configured to acquire, based on vision and three-dimensional depth perception, multi-modal perception information of a target scene, wherein the multi-modal perception information comprises a Red-Green-Blue (RGB) visual image and depth perception information, and the depth perception information comprises an ordered point cloud and a disordered point cloud;
a processor;
and
a memory having programming instructions stored thereon, which, when executed by the processor, cause the processor to:
acquire a regular parameterized line set of the visual image through an Ellipse and Line Segment Detector with Continuous validation (ELSDc), and embed and encode line segments and ellipse arcs in the regular parameterized line set to obtain an embedding vector with a length of 256 dimensions;
perform visual feature extraction on the visual image through an image feature extraction module;
input the embedding vector and visual features into a trained parameterized line discrimination network model to obtain a structural semantic line set;
preprocess the ordered point cloud and the disordered point cloud in the depth perception information, and associate two-dimensional structural lines in the structural semantic line set with pixels of the visual image by indexing structural semantic line segments of the ordered point cloud and the disordered point cloud and passed pixels, to obtain an association relationship between the two-dimensional structural lines and three-dimensional points;
perform three-dimensional space fitting on the two-dimensional structural lines based on the association relationship between the two-dimensional structural lines and the three-dimensional points to obtain three-dimensional structural lines;
perform geometric primitive half-plane search based on the three-dimensional structural lines, and determine whether point clouds on both sides of each three-dimensional structural line satisfy a constraint of a regular surface, if so, perform geometric primitive neighborhood expansion according to a greedy algorithm, and determine a maximum regular area;
and
fuse the determined maximum regular area with a point cloud set to construct structural map data.

9 more elements

Dependencies are inferred from the claim text.