# US 2015/0102945 A1 · Multi-agent reinforcement learning for integrated and networked adaptive traffic signal control

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2015/0102945 A1
- Document ID: US20150102945A1
- Status: Published application
- Kind: A1
- Application number: 14/364,998
- Filed: 2012-12-10
- Published: 2015-04-16
- Earliest claimed: 2011-12-16
- Assignees: Pragmatek Transport Innovations, Inc.
- Applicants: Pragmatek Transport Innovations, Inc.
- Inventors: Samah El-Tantawy; Baher Abdulhai
- Web page: https://patentgrep.com/patent/US20150102945A1

## CPC

- G08G 1/081: Plural intersections under common control
- G08G 1/083: Controlling the allocation of time between phases of a cycle
- G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93})

## Abstract

A system and method of multi-agent reinforcement learning for integrated and networked adaptive traffic controllers (MARLIN-ATC). Agents linked to traffic signals generate control actions for an optimal control policy based on traffic conditions at the intersection and one or more other intersections. The agent provides a control action considering the control policy for the intersection and one or more neighbouring intersections. Due to the cascading effect of the system, each agent implicitly considers the whole traffic environment, which results in an overall optimized control policy.

## Claims

### Claim 1

A system for adaptive traffic signal control comprising an agent associated with a traffic signal array, the agent operable to generate a control action for the traffic signal array by determining a joint control policy with one or more selected neighbouring traffic signals.

### Claim 2 (depends on 1)

The system of claim 1, wherein the one or more selected neighbouring traffic signals comprise traffic signals adjacent to the traffic signal array.

### Claim 3 (depends on 1)

The system of claim 1, wherein the agent adapts the joint control policy to stochastic traffic patterns.

### Claim 4 (depends on 1)

The system of claim 1, wherein the joint control policy coordinates the traffic signal array with the neighbouring traffic signals.

### Claim 5 (depends on 1)

The system of claim 1, wherein the agent is in communication with similarly configured agents of the selected neighbouring traffic signals.

### Claim 6 (depends on 1)

The system of claim 1, wherein the agent models a policy of each of the selected neighbouring traffic signals.

### Claim 7 (depends on 1)

The system of claim 1, wherein the joint control policy determination comprises observing local traffic conditions at the traffic signal array.

### Claim 8 (depends on 7)

The system of claim 7, wherein the joint control policy determination further comprises observing local traffic conditions at the selected neighbouring traffic signals.

### Claim 9 (depends on 1)

The system of claim 1, wherein the joint control policy determination comprises the application of game theory.

### Claim 10 (depends on 1)

The system of claim 1, wherein the agent and the one or more selected neighbouring traffic signals enable synchronization in a two dimensional network.

### Claim 11

A method for adaptive traffic signal control comprising generating, by an agent comprising a processor, a control action for a traffic signal array associated with the agent by determining a joint control policy with one or more selected neighbouring traffic signals.

### Claim 12 (depends on 11)

The method of claim 11, wherein the one or more selected neighbouring traffic signals comprise traffic signals adjacent to the traffic signal array.

### Claim 13 (depends on 11)

The method of claim 11, further comprising adapting the joint control policy to stochastic traffic patterns.

### Claim 14 (depends on 11)

The method of claim 11, wherein the joint control policy coordinates the traffic signal array with the neighbouring traffic signals.

### Claim 15 (depends on 11)

The method of claim 11, wherein the agent is in communication with similarly configured agents of the selected neighbouring traffic signals.

### Claim 16 (depends on 11)

The method of claim 11, further comprising modelling a policy of each of the selected neighbouring traffic signals.

### Claim 17 (depends on 11)

The method of claim 11, wherein the joint control policy determination comprises observing local traffic conditions at the traffic signal array.

### Claim 18 (depends on 17)

The method of claim 17, wherein the joint control policy determination further comprises observing local traffic conditions at the selected neighbouring traffic signals.

### Claim 19 (depends on 11)

The method of claim 11, wherein the joint control policy determination comprises the application of game theory.

### Claim 20 (depends on 11)

The method of claim 11, wherein the agent and the one or more selected neighbouring traffic signals enable synchronization in a two dimensional network.
