# US 2021/0231774 A1 · Annotation of Radar-Profiles of Objects

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0231774 A1
- Document ID: US20210231774A1
- Status: Published application
- Kind: A1
- Application number: 17/231,685
- Filed: 2021-04-15
- Published: 2021-07-29
- Earliest claimed: 2018-08-29
- Assignees: Aptiv Technologies AG
- Applicants: Aptiv Technologies Limited
- Inventors: Michael H. Laur; Indu Vijayan; Divya Balachandran; Nandita Mangal
- Web page: https://patentgrep.com/patent/US20210231774A1

## CPC

- G01S 7/411: {Identification of targets based on measurements of radar reflectivity (G01S7/415 takes precedence)}
- G01S 13/58: Velocity or trajectory determination systems; Sense-of-movement determination systems
- G01S 13/867: {Combination of radar systems with cameras}
- G01S 13/931: of land vehicles
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES

## Abstract

A radar-data collection system a radar, a camera, and a controller-circuit. The radar and the camera are intended for mounting on a host-vehicle. The radar is configured is to indicate a radar-profile of an object detected by the radar. The camera is configured to render an image of the object. The controller-circuit is in communication with the radar and the camera. The controller is configured to determine an identity of the object in accordance with the image, and annotate the radar-profile in accordance with the identity.

## Claims

### Claim 1

A system, comprising:
a controller circuit configured to:
receive sensor data that captures an object proximate a vehicle, the sensor data comprising image data and lidar data;
determine, based on the lidar data and an image of the object from the image data, a sensor profile including a multi-dimensional representation of the sensor data relative to a coordinate system of the vehicle;
annotate the sensor profile by at least labeling the multi-dimensional representation of the sensor data with the image of the object in addition to an orientation of the object, the orientation of the object being derived from the image data;
responsive to determining a distance and a direction to the object based on the sensor profile, further annotate the sensor profile by further labeling the multi-dimensional representation of the sensor data with the distance and the direction to the object.

### Claim 2 (depends on 1)

The system in accordance with claim 1, wherein the controller circuit is configured to further annotate the sensor profile by at least labeling the multi-dimensional representation of the sensor data with an object classification identifying whether the object comprises another vehicle.

### Claim 3 (depends on 1)

The system in accordance with claim 1,
wherein the controller is further configured to determine a portion of the sensor data that comprises radar data that captures the object;
wherein the controller is further configured to determine the sensor profile including the multi-dimensional representation of the sensor data based on the radar data.

### Claim 4 (depends on 3)

The system in accordance with claim 3, wherein the controller is further configured to:
further annotate the sensor profile by labeling the multi-dimensional representation of the sensor data with an orientation angle of the object, the orientation angle of the object being derived from the radar data; and
responsive to determining, based at least in part on the orientation angle, a distance and a direction to the object, further annotate the sensor profile by labeling the multi-dimensional representation of the sensor data with the distance and the direction to the object.

### Claim 5 (depends on 3)

The system in accordance with claim 3, wherein the controller is further configured to:
receive additional radar data and additional lidar data that each capture another object proximate the vehicle;
determine, based on the additional radar data and the additional lidar data, another sensor profile including another multi-dimensional representation of additional sensor data relative to the coordinate system of the vehicle, the additional sensor data including the additional radar data and the additional lidar data;
annotate the other sensor profile by at least labeling the other multi-dimensional representation of the additional sensor data with an image of the other object and an orientation of the other object, the image of the other object and the orientation of the other object being derived from the additional lidar data;
responsive to determining a distance and a direction to the other object based on the other sensor profile, further annotate the other sensor profile by further labeling the other multi-dimensional representation of the additional sensor data with the distance and the direction to the object.

### Claim 6 (depends on 3)

The system in accordance with claim 3, wherein the controller circuit is further configured to:
determine a reflection intensity map in accordance with the sensor profile; and
further annotate the sensor profile in accordance with the reflection intensity map.

### Claim 7 (depends on 3)

The system in accordance with claim 3, wherein the controller circuit is configured to:
determine, based on the radar data, an elevation to the object in accordance with the sensor profile; and
further annotate the sensor profile by labeling the multi-dimensional representation of the sensor data with the elevation to the object.

### Claim 8 (depends on 1)

The system in accordance with claim 1, wherein the controller is further configured to:
determine, based on the sensor data, a pose angle of the vehicle; and
further annotate the sensor profile by labeling the multi-dimensional representation of the sensor data with the pose angle of the vehicle.

### Claim 9 (depends on 1)

The system in accordance with claim 1, wherein the controller circuit is further configured to operate the vehicle in accordance with the sensor profile in response to a determination that a sensor map corresponds to the sensor profile.

### Claim 10 (depends on 1)

The system in accordance with claim 1, wherein the controller circuit is configured to upload the sensor profile to a data base for use in operating one or more other vehicles in accordance with the sensor profile.

### Claim 11

A computer-readable storage medium comprising instructions that, when executed, configure a controller circuit to:
receive sensor data that captures an object proximate a vehicle;
determine, based on the sensor data, a sensor profile including a multi-dimensional representation of sensor data relative to a coordinate system of the vehicle;
annotate the sensor profile by at least labeling the multi-dimensional representation of the sensor data with an object classification identifying whether the object comprises another vehicle, the object classification being derived from a portion of the sensor data comprising lidar data or image data;
further annotate the sensor profile by further labeling the multi-dimensional representation of the sensor data with a distance and a direction to the object, the distance and the direction to the object being derived from a portion of the sensor data comprising radar data; and
operate the vehicle in accordance with the sensor profile in response to a determination that a sensor map corresponds to the sensor profile.

### Claim 12 (depends on 11)

The computer-readable storage medium of claim 11, wherein the instructions, when executed, further configure the controller circuit to:
further annotate the sensor profile by at least further labeling the multi-dimensional representation of the sensor data with an image of the object in addition to an orientation of the object, the image of the object and the orientation of the object being derived from a portion of the sensor data comprising the lidar data or image data.

### Claim 13

A method, comprising:
receiving, by a controller circuit of a vehicle, sensor data that captures an object proximate the vehicle, the sensor data comprising image data and lidar data;
determining, based on the lidar data and an image of the object from the image data, a sensor profile including a multi-dimensional representation of the sensor data relative to a coordinate system of the vehicle;
annotating the sensor profile by at least labeling the multi-dimensional representation of the sensor data with the image of the object in addition to an orientation of the object, the orientation of the object being derived from the image data;
responsive to determining a distance and a direction to the object based on the sensor profile, further annotating, by the controller circuit, the sensor profile by further labeling the multi-dimensional representation of the sensor data with the distance and the direction to the object.

### Claim 14 (depends on 13)

The method of claim 13, further comprising:
further annotating the sensor profile by at least labeling the multi-dimensional representation of the sensor data with an object classification identifying whether the object comprises another vehicle.

### Claim 15 (depends on 13)

The method of claim 13, further comprising:
determining, by the controller circuit, a portion of the sensor data that comprises radar data that captures the object,
wherein determining the sensor profile including the multi-dimensional representation of the sensor data comprises determining the sensor profile based at least in part on the radar data.

### Claim 16 (depends on 15)

The method of claim 15, further comprising:
further annotating the sensor profile by labeling the multi-dimensional representation of the sensor data with an orientation angle of the object, the orientation angle of the object being derived from the radar data; and
responsive to determining, based at least in part on the orientation angle, a distance and a direction to the object, further annotating the sensor profile by labeling the multi-dimensional representation of the sensor data with the distance and the direction to the object.

### Claim 17 (depends on 15)

The method of claim 15, further comprising:
receiving additional radar data and additional lidar data that each capture another object proximate the vehicle;
determining, based on the additional radar data and the additional lidar data, another sensor profile including another multi-dimensional representation of additional sensor data relative to the coordinate system of the vehicle, the additional sensor data including the additional radar data and the additional lidar data;
annotating the other sensor profile by at least labeling the other multi-dimensional representation of the additional sensor data with an image of the other object and an orientation of the other object, the image of the other object and the orientation of the other object being derived from the additional lidar data;
responsive to determining a distance and a direction to the other object based on the other sensor profile, further annotating the other sensor profile by further labeling the other multi-dimensional representation of the additional sensor data with the distance and the direction to the object.

### Claim 18 (depends on 15)

The method of claim 15, further comprising:
determining a reflection intensity map in accordance with the sensor profile; and
further annotating the sensor profile in accordance with the reflection intensity map.

### Claim 19 (depends on 15)

The method of claim 15, further comprising:
operating the vehicle in accordance with the sensor profile in response to determining that a sensor map corresponds to the sensor profile.

### Claim 20 (depends on 15)

The method of claim 15, further comprising:
uploading the sensor profile to a database for use in operating one or more other vehicles in accordance with the sensor profile.
