# US 2024/0005671 A1 · Vision-Based Indicator Signal Detection Using Spatiotemporal Filtering

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0005671 A1
- Document ID: US20240005671A1
- Status: Published application
- Kind: A1
- Application number: 18/469,955
- Filed: 2023-09-19
- Published: 2024-01-04
- Earliest claimed: 2013-06-26
- Assignees: Waymo LLC
- Applicants: Waymo LLC
- Inventors: Abhijit Ogale; David Ian Franklin Ferguson
- Web page: https://patentgrep.com/patent/US20240005671A1

## CPC

- G06V 20/584: {of vehicle lights or traffic lights}
- B60W 60/001: {Planning or execution of driving tasks}
- G05D 1/0246: {using a video camera in combination with image processing means}
- G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle
- H04N 23/73: by influencing the exposure time
- B60R 2300/80: characterised by the intended use of the viewing arrangement
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- B60R: VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR

## Abstract

An autonomous vehicle is configured to detect an active turn signal indicator on another vehicle. An image-capture device of the autonomous vehicle captures an image of a field of view of the autonomous vehicle. The autonomous vehicle captures the image with a short exposure to emphasize objects having brightness above a threshold. Additionally, a bounding area for a second vehicle located within the image is determined. The autonomous vehicle identifies a group of pixels within the bounding area based on a first color of the group of pixels. The autonomous vehicle also calculates an oscillation of an intensity of the group of pixels. Based on the oscillation of the intensity, the autonomous vehicle determines a likelihood that the second vehicle has a first active turn signal. Additionally, the autonomous vehicle is controlled based at least on the likelihood that the second vehicle has a first active turn signal.

## Claims

### Claim 1

A method comprising:
receiving, at a computing device and from a camera coupled to a first vehicle, image data representing an environment of the first vehicle;
identifying, by the computing device and using the image data, a first group of pixels representing a first signal light of a second vehicle and a second group of pixels representing a second signal light of the second vehicle;
determining a control strategy based on the first group of pixels and the second group of pixels; and
causing, by the computing device, the first vehicle to navigate based on the control strategy.

### Claim 2 (depends on 1)

The method of claim 1, wherein identifying the first group of pixels representing the first signal light of the second vehicle and the second group of pixels representing the second signal light of the second vehicle comprises:
identifying each group of pixels based a predefined template indicative of a turn signal.

### Claim 3 (depends on 2)

The method of claim 2, wherein the predefined template represents both a color and a shape indicative of the turn signal.

### Claim 4 (depends on 1)

The method of claim 1, wherein identifying the first group of pixels representing the first signal light of the second vehicle and the second group of pixels representing the second signal light of the second vehicle comprises:
applying, using the image data, a bounding box around at least a portion of the second vehicle; and
identifying the first group of pixels and the second group of pixels based on the bounding box.

### Claim 5 (depends on 1)

The method of claim 1, wherein identifying the first group of pixels representing the first signal light of the second vehicle and the second group of pixels representing the second signal light of the second vehicle comprises:
identifying the first group of pixels based on the first group of pixels being located in a left half of the second vehicle; and
identifying the second group of pixels based on the second group of pixels being located in a right half of the second vehicle.

### Claim 6 (depends on 1)

The method of claim 1, wherein determining the control strategy based on the first group of pixels and the second group of pixels comprises:
performing a comparison between a first brightness of the first group of pixels and a second brightness of the second group of pixels; and
determining, based on the comparison, that the second vehicle has an active turn signal.

### Claim 7 (depends on 1)

The method of claim 1, wherein determining the control strategy based on the first group of pixels and the second group of pixels comprises:
applying, using the image data, a weighted template to assign a first score to the first group of pixels and a second score to the second group of pixels;
performing a comparison between the first score and the second score; and
determining the control strategy based on the comparison.

### Claim 8 (depends on 7)

The method of claim 7, further comprising:
determining the control strategy based on a difference between the first score and the second score, wherein the difference indicates that the second vehicle has an active turn signal.

### Claim 9 (depends on 7)

The method of claim 7, further comprising:
determining the control strategy based on the first score matching the second score.

### Claim 10 (depends on 1)

The method of claim 1, wherein receiving image data from the camera coupled to the first vehicle comprises:
receiving a plurality of images from a front-facing camera coupled to a front portion of the first vehicle.

### Claim 11 (depends on 1)

The method of claim 1, wherein determining the control strategy based on the first group of pixels and the second group of pixels comprises:
performing a comparison between a first oscillation frequency of the first group of pixels and a second oscillation frequency of the second group of pixels; and
determining the control strategy based on the comparison.

### Claim 12 (depends on 11)

The method of claim 11, wherein determining the control strategy based on the comparison comprises:
determining, based on the comparison, that the first oscillation frequency of the first group of pixels matches the second oscillation frequency of the second group of pixels; and
determining the control strategy based on the second vehicle displaying emergency flashers.

### Claim 13 (depends on 11)

The method of claim 11, wherein determining the control strategy based on the comparison comprises:
determining, based on the comparison, that the first oscillation frequency of the first group of pixels is greater than the second oscillation frequency of the second group of pixels; and
determining the control strategy based on determining that the second vehicle has an active turn signal.

### Claim 14 (depends on 1)

The method of claim 1, wherein determining the control strategy based on the first group of pixels and the second group of pixels comprises:
performing a comparison between a first color channel of the first group of pixels and a second color channel of the second group of pixels; and
determining the control strategy based on the comparison.

### Claim 15 (depends on 14)

The method of claim 14, wherein determining the control strategy based on the comparison comprises:
determining that the first color channel of the first group of pixels matches the second color channel of the second group of pixels; and
determining the control strategy based on the first color channel matching the second color channel.

### Claim 16 (depends on 14)

The method of claim 14, wherein determining the control strategy based on the comparison comprises:
determining that the first color channel of the first group of pixels differs from the second color channel of the second group of pixels; and
determining the control strategy based on the first color channel differing from the second color channel.

### Claim 17

A system comprising:
a camera coupled to a first vehicle, wherein the camera is configured to capture image data of an environment of the first vehicle; and
a computing device configured to:
receive, from the camera, image data representing the environment of the first vehicle;
identify, using the image data, a first group of pixels representing a first signal light of a second vehicle and a second group of pixels representing a second signal light of the second vehicle;
determine a control strategy based on the first group of pixels and the second group of pixels; and
cause the first vehicle to navigate based on the control strategy.

### Claim 18 (depends on 17)

The system of claim 17, wherein the computing device is further configured to:
perform a comparison between a first color corresponding to the first group of pixels and a second color corresponding to the second group of pixels; and
determine the control strategy based on the comparison.

### Claim 19 (depends on 18)

The system of claim 18, wherein the computing device is further configured to:
determine, based on the comparison, that the second vehicle has an active turn signal, wherein the comparison indicates a difference between the first color corresponding to the first group of pixels and the second color corresponding to the second group of pixels.

### Claim 20

A non-transitory computer-readable medium configured to store instructions, that when executed by a computing system, causes the computing system to perform operations comprising:
receiving, from a camera coupled to a first vehicle, image data representing an environment of the first vehicle;
identifying, using the image data, a first group of pixels representing a first signal light of a second vehicle and a second group of pixels representing a second signal light of the second vehicle;
determining a control strategy based on the first group of pixels and the second group of pixels; and
causing the first vehicle to navigate based on the control strategy.
