# US 2021/0241616 A1 · Method and system for multimodal deep traffic signal control

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0241616 A1
- Document ID: US20210241616A1
- Status: Published application
- Kind: A1
- Application number: 17/049,236
- Filed: 2019-04-17
- Published: 2021-08-05
- Earliest claimed: 2018-04-20
- Assignees: The Governing Council of the University of Toronto
- Applicants: The Governing Council of the University of Toronto
- Inventors: Baher ABDULHAI; Soheil Mohamad Alizadeh SHABESTARY
- Web page: https://patentgrep.com/patent/US20210241616A1

## CPC

- G08G 1/0129: {for creating historical data or processing based on historical data}
- G08G 1/0145: {for active traffic flow control}
- G06N 3/006: based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06N 3/092: Reinforcement learning
- G08G 1/0112: {from the vehicle, e.g. floating car data [FCD]}
- G08G 1/0116: {from roadside infrastructure, e.g. beacons}
- G08G 1/0133: {for classifying traffic situation}
- G08G 1/052: with provision for determining speed or overspeed {(speed measuring in general G01P)}
- G08G 1/08: according to detected number or speed of vehicles
- G08G 1/095: Traffic lights
- G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93})
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

There is provided a system and method for traffic signal control for an intersection of a traffic network. The method includes: receiving sensor readings including a plurality of physical characteristics associated with vehicles approaching the intersection; discretizing the sensor readings based on a grid of cells; associating a value representing the physical characteristic for each of the cells; generating a matrix associated with the physical characteristic; combining each matrix associated with each of the plurality of physical characteristics as separate layers in a multi-layered matrix; determining, using a machine learning model trained with a traffic control training set, one or more traffic actions with the multi-layered matrix as input, the traffic control training set including previously determined multi-layered matrices for a plurality of traffic scenarios at the intersection; and communicating the one or more actions to the traffic network.

## Claims

### Claim 1

A method for traffic signal control for an intersection of a traffic network, the traffic network comprising one or more sensors, the method comprising:
receiving sensor readings from the one or more sensors, the sensor readings comprising a plurality of physical characteristics associated with vehicles approaching the intersection;
discretizing the sensor readings based on a grid of cells projected onto one or more streets approaching the intersection;
for each of the plurality of physical characteristics, associating, for each of the cells in the grid of cells, a respective value for the cell in the grid of cells representing the physical characteristic associated with each of the vehicles if the vehicles at least partially occupy the cell, otherwise associating a null value for the cell, and generating a matrix associated with the physical characteristic comprising the respective values for each cell in the grid of cells;
combining each matrix associated with each of the plurality of physical characteristics as separate layers in a multi-layered matrix;
determining, using a machine learning model trained with a traffic control training set, one or more traffic actions with the multi-layered matrix as input, the traffic control training set comprising previously determined multi-layered matrices for a plurality of traffic scenarios at the intersection; and
communicating the one or more actions to the traffic network.

### Claim 2 (depends on 1)

The method of claim 1, wherein one of the physical characteristics is speed of the vehicles and another one of the physical characteristics is position of the vehicles.

### Claim 3 (depends on 1)

The method of claim 1, wherein one of the physical characteristics is occupancy of the vehicles.

### Claim 4 (depends on 3)

The method of claim 3, wherein data representing the occupancy of the vehicle is approximated using an average occupancy for each type of vehicle.

### Claim 5 (depends on 3)

The method of claim 3, wherein at least one of the vehicles is a transit vehicle, and wherein the sensor associated with the occupancy of the vehicle comprises an automated passenger counter associated with the transit vehicle.

### Claim 6 (depends on 1)

The method of claim 1, wherein the machine learning model comprises a convolutional neural network and reinforcement learning.

### Claim 7 (depends on 6)

The method of claim 6, wherein the machine learning model comprises Q-learning by iteratively updating a Q-value function, and wherein the determination of the one or more traffic actions is determined as the traffic actions that have the highest Q-values.

### Claim 8 (depends on 6)

The method of claim 6, wherein the machine learning model is used to optimize a reward function by minimizing cumulative delay of the vehicles approaching the intersection, the reward function comprising cumulative delay at a previous iteration minus cumulative delay at a present iteration.

### Claim 9 (depends on 8)

The method of claim 8, wherein the cumulative delay is determined as a summation of delays over each possible movement of the vehicles in each approach of the intersection.

### Claim 10 (depends on 9)

The method of claim 9, wherein the vehicles are considered in delayed if their speed is below a predetermined speed threshold.

### Claim 11

A system for traffic signal control for an intersection of a traffic network, the traffic network comprising one or more sensors, the system comprising one or more processors and a data storage, the one or more processors configurable to execute:
a data extraction module to:
receive sensor readings from the one or more sensors, the sensor readings comprising a plurality of physical characteristics associated with vehicles approaching the intersection;
discretize the sensor readings based on a grid of cells projected onto one or more streets approaching the intersection;
for each of the plurality of physical characteristics, associate, for each of the cells in the grid of cells, a respective value for the cell in the grid of cells representing the physical characteristic associated with each of the vehicles if the vehicles at least partially occupy the cell, otherwise associating a null value for the cell, and generate a matrix associated with the physical characteristic comprising the respective values for each cell in the grid of cells;
a machine learning module to combine each matrix associated with each of the plurality of physical characteristics as separate layers in a multi-layered matrix, and to determine, using a machine learning model trained with a traffic control training set, one or more traffic actions with the multi-layered matrix as input, the traffic control training set comprising previously determined multi-layered matrices for a plurality of traffic scenarios at the intersection; and
a controller module to communicate the one or more actions to the traffic network.

### Claim 12 (depends on 11)

The system of claim 11, wherein one of the physical characteristics is speed of the vehicles and another one of the physical characteristics is position of the vehicles.

### Claim 13 (depends on 12)

The system of claim 12, wherein one of the physical characteristics is occupancy of the vehicles.

### Claim 14 (depends on 13)

The system of claim 13, wherein data representing the occupancy of the vehicle is approximated using an average occupancy for each type of vehicle.

### Claim 15 (depends on 13)

The system of claim 13, wherein at least one of the vehicles is a transit vehicle, and wherein the sensor associated with the occupancy of the vehicle comprises an automated passenger counter associated with the transit vehicle.

### Claim 16 (depends on 11)

The system of claim 11, wherein the machine learning model comprises a convolutional neural network and reinforcement learning.

### Claim 17 (depends on 16)

The system of claim 16, wherein the machine learning model comprises Q-learning by iteratively updating a Q-value function, and wherein the determination of the one or more traffic actions is determined as the traffic actions that have the highest Q-values.

### Claim 18 (depends on 16)

The system of claim 16, wherein the machine learning model is used to optimize a reward function by minimizing cumulative delay of the vehicles approaching the intersection, the reward function comprising cumulative delay at a previous iteration minus cumulative delay at a present iteration.

### Claim 19 (depends on 18)

The system of claim 18, wherein the cumulative delay is determined as a summation over possible movements of delays over each possible movement of the vehicles in each approach of the intersection.

### Claim 20 (depends on 19)

The system of claim 19, wherein the vehicles are considered delayed if their speed is below a predetermined speed threshold.