# US 10,503,174 B1 · Method and device for optimized resource allocation in autonomous driving on the basis of reinforcement learning using data from lidar, radar, and camera sensor

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,503,174 B1
- Document ID: US10503174B1
- Status: Granted
- Kind: B1
- Application number: 16/263,511
- Filed: 2019-01-31
- Granted: 2019-12-10
- Earliest claimed: 2019-01-31
- Assignees: StradVision, Inc.
- Applicants: Stradvision, Inc.
- Inventors: Kye-Hyeon Kim; Yongjoong Kim; Insu Kim; Hak-Kyoung Kim; Woonhyun Nam; SukHoon Boo; Myungchul Sung; Donghun Yeo; Wooju Ryu; Taewoong Jang; Kyungjoong Jeong; Hongmo Je; Hojin Cho
- Web page: https://patentgrep.com/patent/US10503174B1

## CPC

- B60W 60/0015: {specially adapted for safety}
- G06N 20/00: Machine learning
- B60R 11/04: Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
- G01S 13/931: of land vehicles
- G01S 17/931: of land vehicles
- G05D 1/00: Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots (drive control systems specially adapted for autonomous road vehicles B60W60/00)
- G05D 1/024: {in combination with a laser (lasers per se H01S)}
- G05D 1/0248: {in combination with a laser (lasers per se H01S)}
- G05D 1/0251: {extracting 3D information from a plurality of images taken from different locations, e.g. stereo vision (stereoscopic image analysis H04N13/00; depth recovery from images G06T7/593)}
- G06N 3/006: based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/092: Reinforcement learning
- G06T 7/521: from laser ranging, e.g. using interferometry; from the projection of structured light
- G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle
- G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- G01S 13/865: {Combination of radar systems with lidar systems}
- G01S 13/867: {Combination of radar systems with cameras}
- G01S 17/88: Lidar systems specially adapted for specific applications
- G01S 2013/9318: {Controlling the steering}
- G06T 2207/30261: Obstacle
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- B60R: VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A method for efficient resource allocation in autonomous driving by reinforcement learning is provided for reducing computation via a heterogeneous sensor fusion. This attention-based method includes steps of: a computing device instructing an attention network to perform a neural network operation by referring to attention sensor data, to calculate attention scores; instructing a detection network to acquire video data by referring to the attention scores and to generate decision data for the autonomous driving; instructing a drive network to operate the autonomous vehicle by referring to the decision data, to acquire circumstance data, and to generate a reward by referring to the circumstance data; and instructing the attention network to adjust parameters used for the neural network operation by referring to the reward. Thus, a virtual space where the autonomous vehicle optimizes the resource allocation can be provided by the method.

## Claims

### Claim 1

A method for efficient resource allocation in autonomous driving by reinforcement learning, comprising steps of:
(a) a computing device, if at least one attention sensor data is acquired, performing at least one neural network operation by referring to the attention sensor data, to thereby calculate one or more attention scores;
(b) the computing device acquiring at least one video data taken by at least part of one or more cameras installed on an autonomous vehicle by referring to the attention scores and to generate at least one decision data for the autonomous driving by referring to the video data;
(c) the computing device operating the autonomous vehicle by referring to the decision data, to acquire at least one circumstance data representing a change of circumstance of the autonomous vehicle in operation, and to generate at least one reward by referring to the circumstance data; and
(d) the computing device adjusting at least part of one or more parameters used for the neural network operation by referring to the reward.

### Claim 2 (depends on 1)

The method of claim 1, wherein, at the step of (a), the neural network operation includes operations of at least one convolutional layer into which the attention sensor data is inputted, at least one pooling layer, at least one FC layer, and at least one softmax layer from which the attention scores are outputted, to thereby determine at least one level of driving risk per each of angle ranges included in space near the autonomous vehicle.

### Claim 3 (depends on 1)

The method of claim 1, wherein, at the step of (b), the computing device acquires a specific video data of a specific camera, installed on the autonomous vehicle, corresponding to a specific angle range having its corresponding attention score equal to or greater than a first threshold.

### Claim 4 (depends on 1)

The method of claim 1, wherein, at the step of (b), the computing device generates the decision data by using the video data where at least one region, corresponding to a specific angle range having its corresponding attention score equal to or greater than a first threshold, is set as at least one ROI.

### Claim 5 (depends on 1)

The method of claim 1, wherein, at the step of (c), the circumstance data includes at least part of (i) information on whether at least one vehicle horn of at least one nearby vehicle within a certain distance from the autonomous vehicle is in use, (ii) information on a change of speed of said at least one nearby vehicle, and (iii) information on at least one accident between the autonomous vehicle and said at least one nearby vehicle.

### Claim 6 (depends on 1)

The method of claim 1, wherein, at the step of (a), the attention sensor data is acquired by using one of a radar, a LiDAR, and both of the radar and the LiDAR, and
wherein, if the attention sensor data is acquired by the LiDAR or by both of the radar and the LiDAR, the computing device generates at least one sparse depth image by referring to three dimensional coordinates included in the attention sensor data, generates at least one dense depth image by applying at least one smoothing operation to the sparse depth image, and performs the neural network operation by referring to the dense depth image, to thereby calculate the attention scores.

### Claim 7 (depends on 1)

The method of claim 1, wherein the autonomous vehicle is implemented in a virtual space and wherein at least part of the parameters are adjusted by a process of driving the autonomous vehicle in the virtual space.

### Claim 8 (depends on 7)

The method of claim 7, wherein variations of the attention sensor data, the video data, and the circumstance data are programmed such that the attention sensor data, the video data, and the circumstance data are acquired by referring to information on relations among one or more virtual objects located within a certain distance from the autonomous vehicle in the virtual space.

### Claim 9 (depends on 7)

The method of claim 7, wherein the virtual space is programmed to (i) make at least one nearby vehicle in the virtual space to generate its horn if the nearby vehicle within a certain distance from the autonomous vehicle is detected as having a possibility of collision with the autonomous vehicle within a first threshold time, and (ii) make the nearby vehicle to reduce its speed if the nearby vehicle is detected as having a possibility of collision with the autonomous vehicle within a second threshold time, and wherein the first threshold time is equal to or greater than the second threshold time.

### Claim 10

A computing device for efficient resource allocation in autonomous driving by reinforcement learning, comprising:
at least one memory that stores instructions; and
at least one processor configured to execute the instructions to: perform processes of (I) performing at least one neural network operation by referring to at least one attention sensor data, to thereby calculate one or more attention scores, (II) acquiring at least one video data taken by at least part of one or more cameras installed on an autonomous vehicle by referring to the attention scores and to generate at least one decision data for the autonomous driving by referring to the video data, (III) operating the autonomous vehicle by referring to the decision data, to acquire at least one circumstance data representing a change of circumstance of the autonomous vehicle in operation, and to generate at least one reward by referring to the circumstance data, and (IV) adjusting at least part of one or more parameters used for the neural network operation by referring to the reward.

### Claim 11 (depends on 10)

The computing device of claim 10, wherein, at the process of (I), the neural network operation includes operations of at least one convolutional layer into which the attention sensor data is inputted, at least one pooling layer, at least one FC layer, and at least one softmax layer from which the attention scores are outputted, to thereby determine at least one level of driving risk per each of angle ranges included in space near the autonomous vehicle.

### Claim 12 (depends on 10)

The computing device of claim 10, wherein, at the process of (II), the processor acquires a specific video data of a specific camera, installed on the autonomous vehicle, corresponding to a specific angle range having its corresponding attention score equal to or greater than a first threshold.

### Claim 13 (depends on 10)

The computing device of claim 10, wherein, at the process of (II), the processor generates the decision data by using the video data where at least one region, corresponding to a specific angle range having its corresponding attention score equal to or greater than a first threshold, is set as at least one ROI.

### Claim 14 (depends on 10)

The computing device of claim 10, wherein, at the process of (III), the circumstance data includes at least part of (i) information on whether at least one vehicle horn of at least one nearby vehicle within a certain distance from the autonomous vehicle is in use, (ii) information on a change of speed of said at least one nearby vehicle, and (iii) information on at least one accident between the autonomous vehicle and said at least one nearby vehicle.

### Claim 15 (depends on 10)

The computing device of claim 10, wherein, at the process of (I), the attention sensor data is acquired by using one of a radar, a LiDAR, and both of the radar and the LiDAR, and
wherein, if the attention sensor data is acquired by the LiDAR or by both of the radar and the LiDAR, the processor generates at least one sparse depth image by referring to three dimensional coordinates included in the attention sensor data, generates at least one dense depth image by applying at least one smoothing operation to the sparse depth image, and performs the neural network operation by referring to the dense depth image, to thereby calculate the attention scores.

### Claim 16 (depends on 10)

The computing device of claim 10, wherein the autonomous vehicle is implemented in a virtual space and wherein at least part of the parameters are adjusted by a process of driving the autonomous vehicle in the virtual space.

### Claim 17 (depends on 16)

The computing device of claim 16, wherein variations of the attention sensor data, the video data, and the circumstance data are programmed such that the attention sensor data, the video data, and the circumstance data are acquired by referring to information on relations among one or more virtual objects located within a certain distance from the autonomous vehicle in the virtual space.

### Claim 18 (depends on 16)

The computing device of claim 16, wherein the virtual space is programmed to (i) make at least one nearby vehicle in the virtual space to generate its horn if the nearby vehicle within a certain distance from the autonomous vehicle is detected as having a possibility of collision with the autonomous vehicle within a first threshold time, and (ii) make the nearby vehicle to reduce its speed if the nearby vehicle is detected as having a possibility of collision with the autonomous vehicle within a second threshold time, and wherein the first threshold time is equal to or greater than the second threshold time.