# US 2025/0095487 A1 · Traffic control method and apparatus with autonomous vehicle based on adaptive reward > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2025/0095487 A1 - Document ID: US20250095487A1 - Status: Published application - Kind: A1 - Application number: 18/404,077 - Filed: 2024-01-04 - Published: 2025-03-20 - Earliest claimed: 2023-09-15 - Assignees: Foundation of Soongsil University-Industry Cooperation - Applicants: Foundation of Soongsil University-Industry Cooperation - Inventors: Min Hae KWON; Chan In EOM; Dong Su LEE - Web page: https://patentgrep.com/patent/US20250095487A1 ## CPC - G08G 1/096725: {where the received information generates an automatic action on the vehicle control} - B60W 30/143: {Speed control (B60W30/16 takes precedence)} - B60W 30/18163: {Lane change; Overtaking manoeuvres} - B60W 60/001: {Planning or execution of driving tasks} - G08G 1/0112: {from the vehicle, e.g. floating car data [FCD]} - G08G 1/0125: {Traffic data processing} - G08G 1/0145: {for active traffic flow control} - G08G 1/096775: {where the origin of the information is a central station} - B60W 2554/802: Longitudinal distance - B60W 2720/103: Speed profile - G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93}) - B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT ## Abstract A traffic control method and apparatus with an autonomous vehicle based on adaptive reward. A traffic control apparatus for an autonomous vehicle based on adaptive reward comprises an information observation unit that collects observation information from a sensing module of an autonomous vehicle or a roadside unit (RSU); a policy execution unit that decides on an action including adjusting acceleration and changing lanes of the autonomous vehicle based on the observation information and policy; and a reward determination unit that determines reward according to observation information at a next timestep according to the decision made, wherein reward in the reward determination unit includes penalty in an event of an accident and reward when driving, wherein the reward when driving includes an adaptive target speed reward term, a successful lane change reward term, and a safety distance compliance reward term that are adaptively determined according to road traffic. ## Claims ### Claim 1 A traffic control apparatus for an autonomous vehicle based on adaptive reward comprising: an information observation unit that collects observation information from a sensing module of an autonomous vehicle or a roadside unit (RSU); a policy execution unit that decides on an action including adjusting acceleration and changing lanes of the autonomous vehicle based on the observation information and policy; and a reward determination unit that determines a reward according to observation information at a next timestep according to the decision made, wherein the reward in the reward determination unit includes a penalty in the event of an accident and a reward when driving, wherein the reward when driving includes an adaptive target speed reward term that is adaptively determined according to road traffic, a successful lane change reward term, and a safety distance compliance reward term. ### Claim 2 (depends on 1) The apparatus of claim 1, wherein the observation information includes at least one of a relative distance vector for each lane between front and rear vehicles and the autonomous vehicle, a relative speed vector for each lane for front and rear vehicles, a persistence vector for each lane ahead within an observable range of the autonomous vehicle, and an absolute speed of the autonomous vehicle. ### Claim 3 (depends on 1) The apparatus of claim 1, wherein the adaptive target speed reward term obtains maximum reward when approaching a target speed, which changes at every timestep based on a speed of a front vehicle of an autonomous vehicle, and obtains penalty when driving at a speed exceeding a speed limit. ### Claim 4 (depends on 3) The apparatus of claim 3, wherein the target speed is determined as a maximum value among speeds of vehicles ahead within an observable range of the autonomous vehicle. ### Claim 5 (depends on 1) The apparatus of claim 1, wherein the safety distance compliance reward term includes a reward term for maintaining a safety distance of the autonomous vehicle itself with respect to a front vehicle in the same lane and a safety distance reward term with respect to a rear vehicle in the same lane. ### Claim 6 (depends on 1) The apparatus of claim 1, wherein training of the policy is performed by the autonomous vehicle itself or by a central server connected to multiple autonomous vehicles through a network. ### Claim 7 (depends on 6) The apparatus of claim 6 further comprises, when the autonomous vehicle performs training of the policy on its own, a policy training unit that updates a policy according to experience information including the collected observation information, current action information according to the observation information, reward information according to the current action information, and observation information at a next timestep after the current action information, and learns a policy by determining whether an accident has occurred and whether the number of training times is met. ### Claim 8 (depends on 6) The apparatus of claim 6 further comprises, when the central server performs training of the policy, a policy communication unit that transmits experience information including the collected observation information, current action information according to the observation information, reward information according to the current action information, and observation information at a next timestep after the current action executed to the central server, and receives a policy learned using experience information received from the multiple autonomous vehicles. ### Claim 9 A traffic control apparatus for an autonomous vehicle based on adaptive reward comprising: a processor; and a memory connected to the processor, wherein the memory includes program instructions that perform operations, when executed by the processor, comprising, collecting observation information from a sensing module of an autonomous vehicle or a roadside unit (RSU), deciding on an action including adjusting acceleration and changing lanes of the autonomous vehicle based on the observation information and policy, determining reward according to observation information at a next timestep according to the decision made, wherein the reward includes a penalty in the event of an accident and a reward when driving, wherein the reward when driving includes an adaptive target speed reward term, a successful lane change reward term, and a safety distance compliance reward term that are adaptively determined according to road traffic. ### Claim 10 A method in an apparatus including a processor and a memory for performing traffic control for an autonomous vehicle based on adaptive reward comprising: collecting observation information from a sensing module of an autonomous vehicle or a roadside unit (RSU); deciding on an action including adjusting acceleration and changing lanes of the autonomous vehicle based on the observation information and policy; and determining reward according to observation information at a next timestep according to the decision made, wherein the reward includes a penalty in the event of an accident and a reward when driving, wherein the reward when driving includes an adaptive target speed reward term, a successful lane change reward term, and a safety distance compliance reward term that are adaptively determined according to road traffic. ### Claim 11 (depends on 10) The method of claim 10, wherein the observation information includes at least one of a relative distance vector for each lane between front and rear vehicles and the autonomous vehicle, a relative speed vector for each lane for front and rear vehicles, a persistence vector for each lane ahead within an observable range of the autonomous vehicle, and an absolute speed of the autonomous vehicle. ### Claim 12 (depends on 10) The method of claim 10, wherein the adaptive target speed reward term obtains maximum reward when approaching a target speed, which changes at every timestep based on a speed of a front vehicle of an autonomous vehicle, and obtains penalty when driving at a speed exceeding a speed limit. ### Claim 13 (depends on 12) The method of claim 12, wherein the target speed is determined as a maximum value among speeds of vehicles ahead within an observable range of the autonomous vehicle.