# US 12,112,624 B2 · Systems and methods for traffic-flow regulation via centralized lateral flow control > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,112,624 B2 - Document ID: US12112624B2 - Status: Granted - Kind: B2 - Application number: 17/698,305 - Filed: 2022-03-18 - Published: 2023-04-27 - Granted: 2024-10-08 - Earliest claimed: 2021-10-21 - Assignees: Toyota Motor Engineering & Manufacturing North America, Inc. - Applicants: Toyota Motor Engineering & Manufacturing North America, Inc. - Inventors: Abdul Rahman Kreidieh; Yashar Zeiynali Farid; Kentaro Oguchi - Web page: https://patentgrep.com/patent/US12112624B2 ## CPC - G08G 1/0141: {for traffic information dissemination} - B60W 30/18163: {Lane change; Overtaking manoeuvres} - G05B 13/0265: {the criterion being a learning criterion} - G06N 3/006: based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO] - G06N 3/044: Recurrent networks, e.g. Hopfield networks - G06N 3/045: Combinations of networks - G06N 3/08: Learning methods - G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks - G08G 1/0116: {from roadside infrastructure, e.g. beacons} - G08G 1/0133: {for classifying traffic situation} - G08G 1/0145: {for active traffic flow control} - G08G 1/096708: {where the received information might be used to generate an automatic action on the vehicle control} - G08G 1/096725: {where the received information generates an automatic action on the vehicle control} - G08G 1/096775: {where the origin of the information is a central station} - G08G 1/096783: {where the origin of the information is a roadside individual element} - G08G 1/167: {Driving aids for lane monitoring, lane changing, e.g. blind spot detection} - H04W 4/46: for vehicle-to-vehicle communication [V2V] - B60W 2552/10: Number of lanes - B60W 2556/45: External transmission of data to or from the vehicle - G06N 20/00: Machine learning - G08G 1/0112: {from the vehicle, e.g. floating car data [FCD]} - G08G 1/0129: {for creating historical data or processing based on historical data} - G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93}) - B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT - G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72) ## Abstract Systems and methods described herein relate to traffic-flow regulation via centralized lateral flow control. One embodiment receives, at a locality manager that regulates traffic flow on a roadway via lateral flow control, aggregated macroscopic traffic state information from a section manager that communicates with one or more connected vehicles in a section of the roadway; processes the aggregated macroscopic traffic state information at the locality manager using a reinforcement-learning-based model to determine target lateral flows for two or more lanes of the roadway in the section of the roadway; and transmits the target lateral flows from the locality manager to the section manager, which converts the target lateral flows to lane-change actions and transmits the lane-change actions to the one or more connected vehicles. ## Claims ### Claim 1 A system for traffic-flow regulation via centralized lateral flow control, the system comprising: a processor; and a memory storing machine-readable instructions that, when executed by the processor, cause the processor to: receive aggregated macroscopic traffic state information from a section manager that communicates with one or more connected vehicles in a section of a roadway; process the aggregated macroscopic traffic state information using a reinforcement-learning-based model to determine target lateral flows for two or more lanes of the roadway in the section of the roadway; transmit the target lateral flows to the section manager, wherein the section manager converts the target lateral flows to lane-change actions and transmits the lane-change actions to the one or more connected vehicles; and control at least one of the one or more connected vehicles in performing a lane-change action, of the lane-change actions. ### Claim 2 (depends on 1) The system of claim 1, wherein the machine-readable instructions to process the aggregated macroscopic traffic state information using the reinforcement-learning-based model to determine the target lateral flows include instructions that, when executed by the processor, cause the processor to output an attractiveness term and to compute the target lateral flow for a lane among the two or more lanes of the roadway from a minimum lateral flow, a maximum lateral flow, and the attractiveness term. ### Claim 3 (depends on 1) The system of claim 1, wherein the target lateral flow for a lane among the two or more lanes of the roadway is one of to the left and to the right. ### Claim 4 (depends on 1) The system of claim 1, wherein the reinforcement-learning-based model is based on a Markov Decision Process. ### Claim 5 (depends on 1) The system of claim 1, wherein a reward function used to train the reinforcement-learning-based model includes a societal-impact parameter that determines an extent to which the one or more connected vehicles are favored over legacy vehicles in achieving a predetermined traffic-flow objective. ### Claim 6 (depends on 5) The system of claim 5, wherein the predetermined traffic-flow objective is average speed. ### Claim 7 (depends on 1) The system of claim 1, wherein the machine-readable instructions include further instructions that, when executed by the processor, cause the processor to: serve a geographical area that includes at least one additional section of the roadway; process additional aggregated macroscopic traffic state information received from at least one additional section manager in the at least one additional section using the reinforcement-learning-based model to determine at least one additional target lateral flow for the two or more lanes in the at least one additional section; and transmit the at least one additional target lateral flow to the at least one additional section manager. ### Claim 8 (depends on 1) The system of claim 1, wherein the one or more connected vehicles are autonomous vehicles. ### Claim 9 A non-transitory computer-readable medium for traffic-flow regulation via centralized lateral flow control and storing instructions that, when executed by a processor, cause the processor to: receive aggregated macroscopic traffic state information from a section manager that communicates with one or more connected vehicles in a section of a roadway; process the aggregated macroscopic traffic state information using a reinforcement-learning-based model to determine target lateral flows for two or more lanes of the roadway in the section of the roadway; transmit the target lateral flows to the section manager, wherein the section manager converts the target lateral flows to lane-change actions and transmits the lane-change actions to the one or more connected vehicles; and control at least one of the one or more connected vehicles in performing a lane-change action, of the lane-change actions. ### Claim 10 (depends on 9) The non-transitory computer-readable medium of claim 9, wherein the instructions to process the aggregated macroscopic traffic state information using the reinforcement-learning-based model to determine the target lateral flows include instructions that, when executed by the processor, cause the processor to output an attractiveness term and to compute the target lateral flow for a lane among the two or more lanes of the roadway from a minimum lateral flow, a maximum lateral flow, and the attractiveness term. ### Claim 11 (depends on 9) The non-transitory computer-readable medium of claim 9, wherein a reward function used to train the reinforcement-learning-based model includes a societal-impact parameter that determines an extent to which the one or more connected vehicles are favored over legacy vehicles in achieving a predetermined traffic-flow objective. ### Claim 12 (depends on 9) The non-transitory computer-readable medium of claim 9, wherein the instructions include further instructions that, when executed by the processor, cause the processor to: serve a geographical area that includes at least one additional section of the roadway; process additional aggregated macroscopic traffic state information received from at least one additional section manager in the at least one additional section using the reinforcement-learning-based model to determine at least one additional target lateral flow for the two or more lanes in the at least one additional section; and transmit the at least one additional target lateral flow to the at least one additional section manager. ### Claim 13 A method, comprising: receiving, at a locality manager that regulates traffic flow on a roadway via lateral flow control, aggregated macroscopic traffic state information from a section manager that communicates with one or more connected vehicles in a section of the roadway; processing the aggregated macroscopic traffic state information at the locality manager using a reinforcement-learning-based model to determine target lateral flows for two or more lanes of the roadway in the section of the roadway; transmitting the target lateral flows from the locality manager to the section manager, wherein the section manager converts the target lateral flows to lane-change actions and transmits the lane-change actions to the one or more connected vehicles; and controlling at least one of the one or more connected vehicles in performing a lane-change action, of the lane-change actions. ### Claim 14 (depends on 13) The method of claim 13, wherein the reinforcement-learning-based model outputs an attractiveness term and the target lateral flow for a lane among the two or more lanes of the roadway is computed from a minimum lateral flow, a maximum lateral flow, and the attractiveness term. ### Claim 15 (depends on 13) The method of claim 13, wherein the target lateral flow for a lane among the two or more lanes of the roadway is one of to the left and to the right. ### Claim 16 (depends on 13) The method of claim 13, wherein the reinforcement-learning-based model is based on a Markov Decision Process. ### Claim 17 (depends on 13) The method of claim 13, wherein a reward function used to train the reinforcement-learning-based model includes a societal-impact parameter that determines an extent to which the one or more connected vehicles are favored over legacy vehicles in achieving a predetermined traffic-flow objective. ### Claim 18 (depends on 17) The method of claim 17, wherein the predetermined traffic-flow objective is average speed. ### Claim 19 (depends on 13) The method of claim 13, wherein the locality manager: serves a geographical area that includes at least one additional section of the roadway; processes additional aggregated macroscopic traffic state information received from at least one additional section manager in the at least one additional section using the reinforcement-learning-based model to determine at least one additional target lateral flow for the two or more lanes in the at least one additional section; and transmits the at least one additional target lateral flow to the at least one additional section manager. ### Claim 20 (depends on 13) The method of claim 13, wherein the one or more connected vehicles are autonomous vehicles.