# US 12,000,958 B2 · Reinforcement learning engine for a radar system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,000,958 B2
- Document ID: US12000958B2
- Status: Granted
- Kind: B2
- Application number: 17/741,374
- Filed: 2022-05-10
- Published: 2023-03-09
- Granted: 2024-06-04
- Earliest claimed: 2018-04-26
- Assignees: Bdcm A2 LLC
- Applicants: Bdcm A2 LLC
- Inventors: Matthew Paul Harrison
- Web page: https://patentgrep.com/patent/US12000958B2

## CPC

- G01S 7/417: {involving the use of neural networks}
- G01S 13/726: {Multiple target tracking}
- G01S 13/865: {Combination of radar systems with lidar systems}
- G01S 13/867: {Combination of radar systems with cameras}
- G01S 13/931: of land vehicles
- G06F 18/217: {Validation; Performance evaluation; Active pattern learning techniques}
- G06F 18/254: {of classification results, e.g. of results related to same input data}
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/08: Learning methods
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Examples disclosed herein relate to an autonomous driving system in a vehicle, including a radar system with a reinforcement learning engine to control a beam steering antenna and identity targets in a path and a surrounding environment of the vehicle, and a sensor fusion module to receive information from the radar system on the identified targets and compare the information received from the radar system to information received from at least one sensor in the vehicle.

## Claims

### Claim 1

A perception module for detection of objects, comprising:
a reinforcement learning engine comprising at least one neural network configured to determine a control action for a sensor;
a multi-object tracker module coupled to the reinforcement learning engine; and
an object list and occupancy map module coupled to the multi-object tracker module;
an input port configured to receive sensor data; and
an output port configured to send sensor control data;
wherein the reinforcement learning engine provides control of the sensor in response to a detection and an identification of a target in a path by the sensor in a surrounding environment.

### Claim 2 (depends on 1)

The perception module of claim 1, wherein the object list and occupancy map module comprises an output port to configured provide identification information to a sensor fusion module.

### Claim 3 (depends on 2)

The perception module of claim 2, wherein the multi-object tracker module identifies targets in response to tracking information received.

### Claim 4 (depends on 1)

The perception module of claim 1, wherein one or more targets from received data is identified via the least one neural network of the reinforcement learning engine.

### Claim 5 (depends on 4)

The perception module of claim 4, wherein the least one neural network of the reinforcement learning engine comprises a convolutional neural network and a decision neural network.

### Claim 6 (depends on 5)

The perception module of claim 5, further comprising:
a decision neural network; and
a convolutional neural network coupled to the decision neural network, wherein the decision neural network is configured to determine a selection of parameters for the sensor based on an output received from the convolutional neural network.

### Claim 7 (depends on 6)

The perception module of claim 6, wherein a sensor parameter is a detection field of view.

### Claim 8 (depends on 6)

The perception module of claim 6, wherein a second sensor parameter is a direction of detection field of view.

### Claim 9 (depends on 8)

The perception module of claim 8, wherein controls to the sensor implement a steering angle.

### Claim 10 (depends on 9)

The perception module of claim 9, wherein the sensor is a radar module.

### Claim 11 (depends on 6)

The perception module of claim 6, wherein the decision neural network is trained on a multi-dimensional data set.

### Claim 12 (depends on 11)

The perception module of claim 11, wherein the multi-dimensional data set is a hypercube.

### Claim 13 (depends on 12)

The perception module of claim 12, further comprising a scoring module adapted to generate a score for each of a plurality of actions.

### Claim 14 (depends on 13)

The perception module of claim 13, wherein the scores are provided to a sensor fusion module.

### Claim 15 (depends on 11)

The perception module of claim 11, wherein the convolutional neural network transforms radar data into output data for detecting targets and identifying targets.

### Claim 16 (depends on 15)

The perception module of claim 15, wherein the output data forms a point-cloud representation as voxels.