Systems and methods for traffic light detection and classification
AssigneeToyota Research Institute, Inc.
InventorsKun-Hsin Chen, Kuan-Hui Lee, Jia-En Pan, Sudeep Pillai
Life of this patent
Estimating the term…
Abstract
A traffic light classification system for a vehicle includes an image capture device to capture an image of a scene that includes a traffic light with multiple light signals, a processor, and a memory communicably coupled to the processor and storing a first neural network module including instructions that when executed by the processor cause the processor to determine, based on inputting the image into a neural network, a semantic keypoint for each light signal in the traffic light, and determine, based on each semantic keypoint, a classification state of each light signal.
Claims20
A traffic light detection and classification system for a vehicle, comprising:
- a processor;
- and
- a memory communicably coupled to the processor and storing:
- a first neural network module including instructions that when executed by the processor cause the processor to:
- 1) receive, from an image capture device a sequence of images of a scene that includes a traffic light with multiple light signals, and 2) determine, based on inputting the sequence of images into a neural network and for each image in the sequence of images, a semantic keypoint for each light signal in the traffic light based at least in part on a confidence heatmap output from the neural network, determine a classification for the semantic keypoint, and determine, based on each semantic keypoint, a classification state of each light signal, wherein the neural network has been trained to predict semantic keypoints, and the classification for the semantic keypoint identifies a state of a corresponding light signal as being on or off;
- and
- a second neural network module including instructions that when executed by the processor cause the processor to determine a second classification for a target light signal of the multiple signal lights, the second classification being associated with a temporal characteristic of the target light signal, wherein the temporal characteristic of the target signal light is that the target signal light is configured to blink.
A method of classifying a traffic light, comprising:
- capturing a sequence of images of a scene that has a traffic light with multiple light signals;
- determining, by inputting the sequence of images into a neural network and for each image in the sequence of images, a semantic keypoint for each light signal in the traffic light based at least in part on a confidence heatmap output from the neural network, wherein the neural network has been trained to predict semantic keypoints;
- determining a classification for the semantic keypoint, wherein the classification for the semantic keypoint identifies a state of a corresponding light signal as being on or off;
- determining, based on each semantic keypoint, a classification state of each light signal;
- and
- determining a second classification for a target light signal of the multiple signal lights, the second classification being associated with a temporal characteristic of the target light signal, wherein the temporal characteristic of the target signal light is that the target signal light is configured to blink.
A non-transitory computer-readable medium for classifying a traffic light, including instructions that, when executed by one or more processors, cause the one or more processors to:
- capture a sequence of images of a scene that has a traffic light with multiple light signals;
- determine, by inputting the sequence of images into a neural network and for each image in the sequence of images, a semantic keypoint for each light signal in the traffic light based at least in part on a confidence heatmap output from the neural network, wherein the neural network has been trained to predict semantic keypoints;
- determine a classification for the semantic keypoint, wherein the classification for the semantic keypoint identifies a state of a corresponding light signal as being on or off;
- determine, based on each semantic keypoint, a classification state of each light signal;
- and
- determine a second classification for a target light signal of the multiple signal lights, the second classification being associated with a temporal characteristic of the target light signal, wherein the temporal characteristic of the target signal light is that the target signal light is configured to blink.
Dependencies are inferred from the claim text.