Method and device for identifying the signaling state of at least one signaling device

A vehicle configured to detect a signaling state of at least one signaling device comprising a traffic light, the vehicle comprising:

• a processor which is configured to execute steps comprising
• obtaining at least one image which comprises an image of the at least one signaling device;
• extracting a region of the at least one image which comprises the image of the at least one signaling device;
• detecting the at least one signaling device within the extracted region of the at least one image;
• and
• detecting the signaling state of the at least one signaling device after detecting the at least one signaling device within the extracted region,
• wherein the extracting of the reason is performed using at least one of color segmentation, a filtering operation, and a Maximally Stable Extremal Regions (MSER) operation,
• wherein the MSER operation comprises localizing a structure of the traffic light and adjusting a bounding box size, and
• wherein the MSER operation comprises selecting two seed points, one of the two seed points being selected based on an output of the color segmentation so that if the output is red, the bounding box is fixed so that a traffic light blob occupies an upper region of the bounding box, if the output is green, the bounding box is fixed so that the traffic sight blob occupies a lower region of the bounding box and if the output is amber or yellow, tire bounding box is fixed so that the traffic light blob occupies a middle region of the bounding box.

A non-transitory computer readable medium comprising program instructions for detecting a signaling state of at least one signaling device, the program instructions when executed causing a computer processor to execute steps comprising:

• obtaining at least one image which comprises an image of the at least one signaling device;
• extracting a region of the at least one image which comprises the image of the at least one signaling device;
• detecting the at least one signaling device within the extracted region of the at least one image;
• and
• detecting the signaling state of the at least one signaling device after detecting the at least one signaling device within the extracted region,
• wherein the extracting of the region is performed using at least one of color segmentation, a filtering operation, and a Maximally Stable Extremal Regions (MSER) operation,
• wherein the MSER operation comprises localizing a structure of the traffic light and adjusting a bounding box size, and
• wherein are MSER operation comprises selecting two seed points, one of are two seed points being selected based on an output of the color segmentation so drat if the output is red, the bounding box is fixed so that a traffic light blob occupies an upper region of the bounding box, if the output is green, the bounding box is fixed so that the traffic light blob occupies a lower region of the bounding box, and if the output is anther or yellow, the bounding box is fixed so that the traffic light blob occupies a middle region of the bounding box.

A method for detecting a signaling state of at least one signaling device comprising a traffic light, the method comprising:

• obtaining at least one image which comprises an image of the at least one signaling device;
• extracting a region of the at least one image which comprises the image of the at least one signaling device;
• detecting the at least one signaling device within the extracted region of the at least one image;
• and
• detecting the signaling state of the at least one signaling device after detecting the at least one signaling device within the extracted region,
• wherein the detecting of the signaling state is performed using a Convolutional Neural Network (CNN),
• wherein the CNN comprises an input layer, a first convolution layer, a first pooling layer, a second convolution layer, and a second pooling layer,
• wherein the CNN further comprises an output inner product layer, a rectified linear function neuron layer, and an output layer mapping to four neurons which correspond to three traffic light states and an ambiguous state.

A vehicle configured to detect a signaling state of at least one signaling device comprising a traffic light, the vehicle comprising:

• a processor which is configured to execute steps comprising
• obtaining at least one image which comprises an image of the at least one signaling device;
• extracting a region of the at least one image which comprises the image of the at least one signaling device;
• detecting the at least one signaling device within the extracted region of the at least one image;
• and
• detecting the signaling state of the at least one signaling device after detecting the at least one signaling device within the extracted region,
• wherein the detecting of the signaling state is performed using a Convolutional Neural Network (CNN),
• wherein the CNN comprises an input layer, a first convolution layer, a first pooling layer, a second convolution layer, and a second pooling layer,
• wherein the CNN further comprises an output inner product layer, a rectified linear function neuron layer, and an output layer mapping to four neurons which correspond to three traffic light states and an ambiguous state.

A non-transitory computer readable medium comprising program instructions for detecting a signaling state of at least one signaling device, the program instructions when executed causing a computer processor to execute steps comprising:

• obtaining at least one image which comprises an image of the at least one signaling device;
• extracting a region of the at least one image which comprises the image of the at least one signaling device;
• detecting the at least one signaling device within the extracted region of the at least one image;
• and
• detecting the signaling state of the at least one signaling device after detecting the at least one signaling device within the extracted region,
• wherein the detecting of the signaling state is performed using a Convolutional Neural Network (CNN),
• wherein the CNN comprises an input layer, a first convolution layer, a first pooling layer, a second convolution layer, and a second pooling layer,
• wherein the CNN further comprises an output inner product layer, a rectified linear function neuron layer, and an output layer mapping to four neurons which correspond to three traffic light states and an ambiguous state.