# US 2023/0056784 A1 · Method for Detecting Obstacle, Electronic Device, and Storage Medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0056784 A1
- Document ID: US20230056784A1
- Status: Published application
- Kind: A1
- Application number: 17/889,091
- Filed: 2022-08-16
- Published: 2023-02-23
- Earliest claimed: 2021-08-17
- Assignees: Beijing Baidu Netcom Science Technology Co., Ltd.
- Applicants: Beijing Baidu Netcom Science Technology Co., Ltd.
- Inventors: Xiao ZHANG; Fangfang DONG; Guowei WAN
- Web page: https://patentgrep.com/patent/US20230056784A1

## CPC

- G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- B60W 60/001: {Planning or execution of driving tasks}
- G01S 7/4802: {using analysis of echo signal for target characterisation; Target signature; Target cross-section}
- G01S 7/4972: {Alignment of sensor}
- G01S 17/931: of land vehicles
- G06V 20/64: Three-dimensional [3D] objects
- B60W 2420/408: Radar; Laser, e.g. lidar
- G06V 10/454: {Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]}
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/82: using neural networks
- G06V 2201/07: Target detection
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES

## Abstract

The disclosure provides a method for detecting an obstacle, an electronic device, and a storage medium, relates to the field of artificial intelligence, and in particular, to the field of automatic driving. A specific implementation solution includes: acquiring point cloud data collected by a lidar sensor during detection of an obstacle; converting the point cloud data into an obstacle feature map, where the obstacle feature map includes a to-be-detected image feature, and the to-be-detected image feature includes an obstacle feature; performing feature extraction on the to-be-detected image feature in the obstacle feature map to obtain the obstacle feature, increasing a weight value in the obstacle feature map of the obstacle feature during the feature extraction, and monitoring the feature extraction based on monitoring data generated by point cloud distribution data corresponding to the point cloud data; and determining a target obstacle according to the obstacle feature.

## Claims

### Claim 1

A method for detecting an obstacle, comprising:
acquiring point cloud data collected by a lidar sensor during detection of an obstacle;
converting the point cloud data into an obstacle feature map, wherein the obstacle feature map comprises a to-be-detected image feature, and the to-be-detected image feature comprises an obstacle feature;
performing feature extraction on the to-be-detected image feature in the obstacle feature map to obtain the obstacle feature, wherein a weight value in the obstacle feature map of the obstacle feature is increased during the feature extraction, and the feature extraction is monitored based on monitoring data generated by point cloud distribution data corresponding to the point cloud data; and
determining a target obstacle according to the obstacle feature.

### Claim 2 (depends on 1)

The method as claimed in claim 1, wherein the performing feature extraction on the to-be-detected image feature in the obstacle feature map to obtain the obstacle feature comprises:
dividing a process of performing the feature extraction on the to-be-detected image feature into an encoding processing stage and a decoding processing stage;
performing primary extraction on the to-be-detected image feature based on an attention mechanism at the encoding processing stage to obtain an intermediate image feature; and
performing secondary extraction on the intermediate image feature under monitoring according to the monitoring data at the decoding processing stage, to obtain the obstacle feature.

### Claim 3 (depends on 2)

The method as claimed in claim 2, wherein the performing primary extraction on the to-be-detected image feature based on an attention mechanism at the encoding processing stage to obtain an intermediate image feature comprises:
determining a sampling structure of different resolutions based on a size of predetermined obstacle, wherein the sampling structure is a regular triangle structure; and
performing downsampling on the to-be-detected image feature based on the sampling structure at the encoding processing stage, and extracting the intermediate image feature from the to-be-detected image feature.

### Claim 4 (depends on 2)

The method as claimed in claim 2, wherein the performing secondary extraction on the intermediate image feature under monitoring according to the monitoring data at the decoding processing stage, to obtain the obstacle feature comprises:
using the monitoring data to add the point cloud distribution data to the obstacle feature map based on a supervised learning method at the decoding processing stage, and performing upsampling on the obstacle feature map of different resolutions to obtain a processed obstacle feature map; and
performing secondary extraction on the intermediate image feature in the processed obstacle feature map, to obtain the obstacle feature.

### Claim 5 (depends on 1)

The method as claimed in claim 1, further comprising:
determining the point cloud distribution data according to obstacle distribution information in the point cloud data;
determining whether there is a corresponding obstacle in the point cloud data in a case that the point cloud data is not a null value; and
generating the monitoring data based on a Gaussian distribution function and the point cloud distribution data when there is the corresponding obstacle in the point cloud data.

### Claim 6 (depends on 1)

The method as claimed in claim 1, wherein the determining a target obstacle according to the obstacle feature comprises:
using the obstacle feature as input data of an obstacle detection model, and receiving an output result of the obstacle detection model; and
determining whether an object corresponding to the obstacle feature is the target obstacle based on the output result.

### Claim 7 (depends on 1)

The method as claimed in claim 1, wherein before the converting the point cloud data into an obstacle feature map, the method further comprises:
dividing the point cloud data into a plurality of lattices according to distance information; and
determining a point cloud range of the point cloud data according to distribution positions of the point cloud data in the lattices.

### Claim 8 (depends on 1)

The method as claimed in claim 1, wherein the converting the point cloud data into an obstacle feature map comprises:
dividing the point cloud data according to pixels to obtain divided point cloud data; and
performing post-processing manners on the divided point cloud data to obtain a plurality of obstacle features.

### Claim 9 (depends on 5)

The method as claimed in claim 5, wherein when a distance between two or more obstacle points is less than a preset value, the maximum monitoring data in the two or more obstacle points is selected as the monitoring data of an area where two or more obstacle points are located.

### Claim 10 (depends on 5)

The method as claimed in claim 5, wherein when two or more obstacle points intersect with each other, the maximum monitoring data in the two or more obstacle points is selected as the monitoring data of an area where two or more obstacle points are located.

### Claim 11

An electronic device, comprising:
at least one processor, and
a memory, communicatively connected to the at least one processor, wherein
the memory stores an instruction executable by the at least one processor, and the instruction is performed by the at least one processor, to cause the at least one processor to perform the obstacle detection method:
acquiring point cloud data collected by a lidar sensor during detection of an obstacle;
converting the point cloud data into an obstacle feature map, wherein the obstacle feature map comprises a to-be-detected image feature, and the to-be-detected image feature comprises an obstacle feature;
performing feature extraction on the to-be-detected image feature in the obstacle feature map to obtain the obstacle feature, wherein a weight value in the obstacle feature map of the obstacle feature is increased during the feature extraction, and the feature extraction is monitored based on monitoring data generated by point cloud distribution data corresponding to the point cloud data; and
determining a target obstacle according to the obstacle feature.

### Claim 12 (depends on 11)

The electronic device as claimed in claim 11, wherein the instruction is performed to cause the at least one processor to perform the obstacle detection method:
dividing a process of performing the feature extraction on the to-be-detected image feature into an encoding processing stage and a decoding processing stage;
performing primary extraction on the to-be-detected image feature based on an attention mechanism at the encoding processing stage to obtain an intermediate image feature; and
performing secondary extraction on the intermediate image feature under monitoring according to the monitoring data at the decoding processing stage, to obtain the obstacle feature.

### Claim 13 (depends on 12)

The electronic device as claimed in claim 12, wherein the instruction is performed to cause the at least one processor to perform the obstacle detection method:
determining a sampling structure of different resolutions based on a size of predetermined obstacle, wherein the sampling structure is a regular triangle structure; and
performing downsampling on the to-be-detected image feature based on the sampling structure at the encoding processing stage, and extracting the intermediate image feature from the to-be-detected image feature.

### Claim 14 (depends on 12)

The electronic device as claimed in claim 12, wherein the instruction is performed to cause the at least one processor to perform the obstacle detection method:
using the monitoring data to add the point cloud distribution data to the obstacle feature map based on a supervised learning method at the decoding processing stage, and performing upsampling on the obstacle feature map of different resolutions to obtain a processed obstacle feature map; and
performing secondary extraction on the intermediate image feature in the processed obstacle feature map, to obtain the obstacle feature.

### Claim 15 (depends on 11)

The electronic device as claimed in claim 11, wherein the instruction is performed to cause the at least one processor to perform the obstacle detection method:
determining the point cloud distribution data according to obstacle distribution information in the point cloud data;
determining whether there is a corresponding obstacle in the point cloud data in a case that the point cloud data is not a null value; and
generating the monitoring data based on a Gaussian distribution function and the point cloud distribution data when there is the corresponding obstacle in the point cloud data.

### Claim 16 (depends on 11)

The electronic device as claimed in claim 11, wherein the instruction is performed to cause the at least one processor to perform the obstacle detection method:
using the obstacle feature as input data of an obstacle detection model, and receiving an output result of the obstacle detection model; and
determining whether an object corresponding to the obstacle feature is the target obstacle based on the output result.

### Claim 17 (depends on 11)

The electronic device as claimed in claim 11, wherein the instruction is performed to cause the at least one processor to perform the obstacle detection method:
dividing the point cloud data into a plurality of lattices according to distance information; and
determining a point cloud range of the point cloud data according to distribution positions of the point cloud data in the lattices.

### Claim 18 (depends on 11)

The electronic device as claimed in claim 11, wherein the instruction is performed to cause the at least one processor to perform the obstacle detection method:
dividing the point cloud data according to pixels to obtain divided point cloud data; and
performing post-processing manners on the divided point cloud data to obtain a plurality of obstacle features.

### Claim 19

A non-transitory computer readable storage medium, storing a computer instruction, wherein the computer instruction is used for a computer to perform the obstacle detection method:
acquiring point cloud data collected by a lidar sensor during detection of an obstacle;
converting the point cloud data into an obstacle feature map, wherein the obstacle feature map comprises a to-be-detected image feature, and the to-be-detected image feature comprises an obstacle feature;
performing feature extraction on the to-be-detected image feature in the obstacle feature map to obtain the obstacle feature, wherein a weight value in the obstacle feature map of the obstacle feature is increased during the feature extraction, and the feature extraction is monitored based on monitoring data generated by point cloud distribution data corresponding to the point cloud data; and
determining a target obstacle according to the obstacle feature.

### Claim 20 (depends on 19)

The non-transitory computer readable storage medium as claimed in claim 19, wherein the computer instruction is used for the computer to perform the obstacle detection method:
dividing a process of performing the feature extraction on the to-be-detected image feature into an encoding processing stage and a decoding processing stage;
performing primary extraction on the to-be-detected image feature based on an attention mechanism at the encoding processing stage to obtain an intermediate image feature; and
performing secondary extraction on the intermediate image feature under monitoring according to the monitoring data at the decoding processing stage, to obtain the obstacle feature.