# US 2022/0032933 A1 · Systems and methods for generating a task offloading strategy for a vehicular edge-computing environment

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0032933 A1
- Document ID: US20220032933A1
- Status: Published application
- Kind: A1
- Application number: 16/944,522
- Filed: 2020-07-31
- Published: 2022-02-03
- Earliest claimed: 2020-07-31
- Assignees: Toyota Motor Engineering & Manufacturing North America, Inc.
- Applicants: Toyota Motor Engineering & Manufacturing North America, Inc.
- Inventors: Haoxin Wang; BaekGyu Kim
- Web page: https://patentgrep.com/patent/US20220032933A1

## CPC

- B60W 50/06: Improving the dynamic response of the control system, e.g. improving the speed of regulation or avoiding hunting or overshoot
- H04W 4/40: for vehicles, e.g. vehicle-to-pedestrians [V2P]
- B60W 30/14: {Adaptive} cruise control
- G01C 21/3804: {Creation or updating of map data}
- G06F 9/4881: {Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues}
- G06F 9/5027: {the resource being a machine, e.g. CPUs, Servers, Terminals}
- G06F 9/5066: {Algorithms for mapping a plurality of inter-dependent sub-tasks onto a plurality of physical CPUs (mappping at compile time, see G06F8/451)}
- G06N 3/006: based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06N 3/088: Non-supervised learning, e.g. competitive learning
- G06N 3/092: Reinforcement learning
- H04L 67/10: in which an application is distributed across nodes in the network (software deployment G06F8/60; multiprogramming arrangements G06F9/46)
- H04L 67/1014: based on the content of a request
- H04W 4/44: for communication between vehicles and infrastructures, e.g. vehicle-to-cloud [V2C] or vehicle-to-home [V2H]
- B60W 2520/06: Direction of travel
- B60W 2520/10: Longitudinal speed
- B60W 2554/80: Spatial relation or speed relative to objects
- B60W 2556/40: High definition maps
- B60W 2556/45: External transmission of data to or from the vehicle
- G06F 2209/509: Offload
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72)
- G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

Systems and methods described herein relate to generating a task offloading strategy for a vehicular edge-computing environment. One embodiment simulates a vehicular edge-computing environment in which one or more vehicles perform computational tasks whose data is partitioned into segments and performs, for each of a plurality of segments, a Deep Reinforcement Learning (DRL) training procedure that includes receiving state-space information regarding the one or more vehicles and one or more intermediate network nodes; inputting the state-space information to a policy network; generating, from the policy network, an action concerning a current segment; and assigning a reward to the policy network for the action in accordance with a predetermined reward function. This embodiment produces, via the DRL training procedure, a trained policy network embodying an offloading strategy for segmentation offloading of computational tasks from vehicles to one or more of an edge server and a cloud server.

## Claims

### Claim 1

A system for generating a task offloading strategy for a vehicular edge-computing environment, comprising:
one or more processors; and
a memory communicably coupled to the one or more processors and storing:
a simulation module including instructions that when executed by the one or more processors cause the one or more processors to deploy a simulation platform that simulates a vehicular edge-computing environment in which one or more vehicles perform computational tasks associated with one or more vehicular applications, wherein data associated with the computational tasks is partitioned into segments, the simulation platform includes a simulated edge server, the simulated edge server includes a policy network, and the policy network includes a neural network; and
a Deep Reinforcement Learning (DRL) module including instructions that when executed by the one or more processors cause the one or more processors to perform, for each of a plurality of segments, a Deep Reinforcement Learning (DRL) training procedure in which the one or more processors:
receive, at the simulated edge server, state-space information regarding the one or more vehicles and one or more intermediate network nodes, the one or more intermediate network nodes including at least one of base stations and roadside units;
input the state-space information to the policy network;
generate, from the policy network, an action concerning a current segment of a particular computational task associated with a particular vehicle among the one or more vehicles, the action including one of offloading the current segment via a specific base station, offloading the current segment via a specific roadside unit, and holding the current segment in a data buffer of the particular vehicle; and
assign a reward to the policy network for the action in accordance with a predetermined reward function;
wherein the instructions in the DRL module cause the one or more processors, via the DRL training procedure, to produce a trained policy network embodying an offloading strategy for segmentation offloading of computational tasks from vehicles to one or more of an edge server and a cloud server.

### Claim 2 (depends on 1)

The system of claim 1, further comprising a communication module including instructions that when executed by the one or more processors cause the one or more processors to transmit a set of parameters specifying the trained policy network to a particular edge server that deploys the offloading strategy.

### Claim 3 (depends on 1)

The system of claim 1, wherein the one or more vehicular applications include one or more of in-vehicle augmented reality, high-definition map generation, intelligent driving, and edge-assisted adaptive cruise control.

### Claim 4 (depends on 1)

The system of claim 1, wherein the neural network is a multi-layer convolutional neural network.

### Claim 5 (depends on 1)

The system of claim 1, wherein the state-space information includes one or more of vehicle speed, vehicle direction, distances to roadside units and base stations, vehicle data buffer status, and loading levels of roadside units and base stations.

### Claim 6 (depends on 1)

The system of claim 1, wherein the predetermined reward function includes a scaled segment size minus a scaled data transmission cost.

### Claim 7 (depends on 1)

The system of claim 1, wherein the simulation platform includes an edge-assisted vehicular application layer, a heterogeneous network layer, an edge computing environment layer, a geographic region layer, and a DRL layer.

### Claim 8

A non-transitory computer-readable medium for generating a task offloading strategy for a vehicular edge-computing environment and storing instructions that when executed by one or more processors cause the one or more processors to:
deploy a simulation platform that simulates a vehicular edge-computing environment in which one or more vehicles perform computational tasks associated with one or more vehicular applications, wherein data associated with the computational tasks is partitioned into segments, the simulation platform includes a simulated edge server, the simulated edge server includes a policy network, and the policy network includes a neural network;
perform, for each of a plurality of segments, a Deep Reinforcement Learning (DRL) training procedure in which the one or more processors:
receive, at the simulated edge server, state-space information regarding the one or more vehicles and one or more intermediate network nodes, the one or more intermediate network nodes including at least one of base stations and roadside units;
input the state-space information to the policy network;
generate, from the policy network, an action concerning a current segment of a particular computational task associated with a particular vehicle among the one or more vehicles, the action including one of offloading the current segment via a specific base station, offloading the current segment via a specific roadside unit, and holding the current segment in a data buffer of the particular vehicle; and
assign a reward to the policy network for the action in accordance with a predetermined reward function; and
produce, via the DRL training procedure, a trained policy network embodying an offloading strategy for segmentation offloading of computational tasks from vehicles to one or more of an edge server and a cloud server.

### Claim 9 (depends on 8)

The non-transitory computer-readable medium of claim 8, wherein the instructions further include instructions that when executed by one or more processors cause the one or more processors to transmit a set of parameters specifying the trained policy network to a particular edge server that deploys the offloading strategy.

### Claim 10 (depends on 8)

The non-transitory computer-readable medium of claim 8, wherein the one or more vehicular applications include one or more of in-vehicle augmented reality, high-definition map generation, intelligent driving, and edge-assisted adaptive cruise control.

### Claim 11 (depends on 8)

The non-transitory computer-readable medium of claim 8, wherein the neural network is a multi-layer convolutional neural network.

### Claim 12 (depends on 8)

The non-transitory computer-readable medium of claim 8, wherein the state-space information includes one or more of vehicle speed, vehicle direction, distances to roadside units and base stations, vehicle data buffer status, and loading levels of roadside units and base stations.

### Claim 13 (depends on 8)

The non-transitory computer-readable medium of claim 8, wherein the predetermined reward function includes a scaled segment size minus a scaled data transmission cost.

### Claim 14

A method of generating a task offloading strategy for a vehicular edge-computing environment, the method comprising:
deploying a simulation platform that simulates a vehicular edge-computing environment in which one or more vehicles perform computational tasks associated with one or more vehicular applications, wherein data associated with the computational tasks is partitioned into segments, the simulation platform includes a simulated edge server, the simulated edge server includes a policy network, and the policy network includes a neural network;
performing, for each of a plurality of segments, a Deep Reinforcement Learning (DRL) training procedure that includes:
receiving, at the simulated edge server, state-space information regarding the one or more vehicles and one or more intermediate network nodes, the one or more intermediate network nodes including at least one of base stations and roadside units;
inputting the state-space information to the policy network;
generating, from the policy network, an action concerning a current segment of a particular computational task associated with a particular vehicle among the one or more vehicles, the action including one of offloading the current segment via a specific base station, offloading the current segment via a specific roadside unit, and holding the current segment in a data buffer of the particular vehicle; and
assigning a reward to the policy network for the action in accordance with a predetermined reward function; and
producing, via the DRL training procedure, a trained policy network embodying an offloading strategy for segmentation offloading of computational tasks from vehicles to one or more of an edge server and a cloud server.

### Claim 15 (depends on 14)

The method of claim 14, further comprising transmitting a set of parameters specifying the trained policy network to a particular edge server that deploys the offloading strategy.

### Claim 16 (depends on 14)

The method of claim 14, wherein the one or more vehicular applications include one or more of in-vehicle augmented reality, high-definition map generation, intelligent driving, and edge-assisted adaptive cruise control.

### Claim 17 (depends on 14)

The method of claim 14, wherein the neural network is a multi-layer convolutional neural network.

### Claim 18 (depends on 14)

The method of claim 14, wherein the state-space information includes one or more of vehicle speed, vehicle direction, distances to roadside units and base stations, vehicle data buffer status, and loading levels of roadside units and base stations.

### Claim 19 (depends on 14)

The method of claim 14, wherein the predetermined reward function includes a scaled segment size minus a scaled data transmission cost.

### Claim 20 (depends on 14)

The method of claim 14, wherein the simulation platform includes an edge-assisted vehicular application layer, a heterogeneous network layer, an edge computing environment layer, a geographic region layer, and a DRL layer.