# US 2021/0343143 A1 · Method and system for traffic light signal detection and usage

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0343143 A1
- Document ID: US20210343143A1
- Status: Published application
- Kind: A1
- Application number: 17/269,364
- Filed: 2019-08-01
- Published: 2021-11-04
- Earliest claimed: 2018-08-22
- Assignees: Starship Technologies Oü
- Applicants: Starship Technologies Oü
- Inventors: Kristjan KORJUS; Sergii KHARAGORGIIEV; Rasmus RAAG
- Web page: https://patentgrep.com/patent/US20210343143A1

## CPC

- G08G 1/095: Traffic lights
- G06N 3/02: Neural networks
- G06V 20/584: {of vehicle lights or traffic lights}
- G08G 1/094: {Hardware aspects; Signal processing or signal properties, e.g. frequency bands}
- G08G 1/097: Supervising of traffic control systems, e.g. by giving an alarm if two crossing streets have green light simultaneously
- G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93})
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A method comprising a data processing device predicting a time for a future state change of a first traffic light; a method comprising a data processing device generating a map of traffic lights, wherein the map of traffic lights comprises a location of at least one localization traffic light, and methods combining these methods, and to corresponding systems.

## Claims

### Claim at position 1

1-15. (canceled)

### Claim 16

A method comprising:
a data processing device predicting a time for a future state change of a first traffic light.

### Claim 17 (depends on 16)

The method according to claim 16, wherein the method comprises:
capturing a plurality of images of the first traffic light; and
the data processing device determining occurring states of the first traffic light based on the plurality of images of the first traffic light; and
the data processing device determining a time of an occurring state change of the first traffic light based on the determined occurred states of the first traffic light,
wherein the time of the occurring state change of the first traffic light is taken into account for predicting the time for the future state change of the first traffic light.

### Claim 18 (depends on 17)

The method according to claim 17, wherein the method comprises:
capturing a plurality of images of a second traffic light; and
the data processing device determining occurring states of the second traffic light based on the plurality of images of the second traffic light; and
the data processing device determining a time of an occurring state change of the second traffic light based on the determined occurred states of the second traffic light,
wherein the time of the occurring state change of the second traffic light is taken into account for predicting the time for the future state change of the first traffic light, and
wherein the first traffic light is located on a first crossing.

### Claim 19 (depends on 18)

The method according to claim 18, wherein the second traffic light is located on the first crossing.

### Claim 20 (depends on 18)

The method according to claim 18, wherein the second traffic light is not located on the first crossing.

### Claim 21 (depends on 16)

The method according to claim 16, wherein the method comprises a first mobile robot driving.

### Claim 22 (depends on 18)

The method according to claim 18,
wherein the method comprises a first mobile robot driving, and
wherein the first mobile robot comprises a first camera system, and
wherein the plurality of images of the first traffic light are captured by the first camera system, and
wherein the method comprises a second mobile robot driving, and
wherein the second mobile robot comprises a second camera system, wherein the plurality of images of the second traffic light are captured by the second camera system.

### Claim 23 (depends on 21)

The method according to claim 21, wherein the future state change of the first traffic light is a state change of the first traffic light from a state prohibiting the first mobile robot to cross a road to a state allowing the first mobile robot to cross the road, and
wherein the method further comprises:
notifying an operator terminal of the future state change before the predicted time of the future state change.

### Claim 24 (depends on 21)

The method according to claim 21, wherein a route of the first mobile robot is altered in response to the predicted time for a future state change.

### Claim 25 (depends on 17)

The method according to claim 17, wherein the method comprises:
determining a first pattern of time delays between occurring state changes of the first traffic light based on the plurality of images of the first traffic light,
wherein the first pattern is taken into account for predicting the time for the future state change of the first traffic light.

### Claim 26 (depends on 25)

The method according to claim 25, wherein the first pattern is time dependent.

### Claim 27 (depends on 16)

A method of generating and utilizing a map of traffic lights, wherein the method comprises:
a data processing device generating a map of traffic lights, wherein the map of traffic lights comprises a location of at least one localization traffic light, and
wherein the method further comprises the method according to claim 16,
wherein the at least one localization traffic light comprises the first traffic light,
wherein the method further comprises adding to the map of traffic lights the location of the first traffic light.

### Claim 28 (depends on 27)

The method according to claim 27, wherein the method comprises:
capturing a plurality of images of the at least one localization traffic light;
determining a capture location where at least one image from the plurality of images of the at least one localization traffic light is captured; and
determining the location of the at least one localization traffic light based on the capture location of the at least one image of the at least one localization traffic light.

### Claim 29 (depends on 28)

The method according to claim 28, wherein the step of capturing a plurality of images of the at least one localization traffic light comprises:
capturing a plurality of images; and
the data processing device determining presence of at least one localization traffic light on each of the plurality of captured images, wherein the step of determining the presence of at least one localization traffic light on each of the plurality of captured images comprises utilizing a machine learning algorithm

### Claim 30 (depends on 29)

The method of claim 29, wherein the machine learning algorithm comprises a neural network algorithm

### Claim 31 (depends on 16)

A system configured to perform the method of claim 16.
