Apparatus, method, and computer program for identifying state of signal light, and controller

A controller for controlling travel of a vehicle, the controller comprising:

• a processor configured to:
• input time series images into a first classifier to detect object regions each including another vehicle equipped with a signal light in the respective images, the first classifier having been trained to detect the other vehicle;
• chronologically input characteristics obtained from pixel values of the object regions detected in the respective images into a second classifier to calculate confidence scores of possible candidate states of the signal light of the other vehicle, the second classifier having a recursive structure or performing a convolution operation in a temporal direction;
• identify the state of the signal light, based on the preceding state of the signal light, information indicating whether transitions between the candidate states are allowed, and the confidence scores of the respective candidate states;
• predict a trajectory on which the other vehicle will travel, based on the state of the signal light;
• determine a trajectory to be traveled of the vehicle, based on the predicted trajectory, so that the vehicle will be separated from the other vehicle more than a predetermined distance;
• and
• control the vehicle so that the vehicle will travel along the trajectory to be traveled.

A method for identifying the state of a signal light, comprising:

• inputting time series images into a first classifier to detect object regions each including a vehicle equipped with a signal light in the respective images, the first classifier having been trained to detect the vehicle;
• chronologically inputting characteristics obtained from pixel values of the object regions detected in the respective images into a second classifier to calculate confidence scores of possible candidate states of the signal light of the vehicle, the second classifier having a recursive structure or performing a convolution operation in a temporal direction;
• and
• identifying the state of the signal light, based on the preceding state of the signal light, information indicating whether transitions between the candidate states are allowed, and the confidence scores of the respective candidate states;
• wherein the information indicates candidate states to which the respective candidate states are allowed to change, and of the candidate states to which the preceding state of the signal light is allowed to change, the processor identifies the state of the signal light as a candidate state having a maximum confidence score;
• and
• wherein the candidate states include a turn-on state in which the signal light is turned on, a turn-off state in which the signal light is turned off, an ON state during which the signal light is on, and an OFF state during which the signal light is off.

A non-transitory recording medium having recorded thereon a computer program for identifying the state of a signal light, the computer program causing a computer to execute a process comprising:

• inputting time series images into a first classifier to detect object regions each including a vehicle equipped with a signal light in the respective images, the first classifier having been trained to detect the vehicle;
• chronologically inputting characteristics obtained from pixel values of the object regions detected in the respective images into a second classifier to calculate confidence scores of possible candidate states of the signal light of the vehicle, the second classifier having a recursive structure or performing a convolution operation in a temporal direction;
• and
• identifying the state of the signal light, based on the preceding state of the signal light, information indicating whether transitions between the candidate states are allowed, and the confidence scores of the respective candidate states;
• wherein the information indicates candidate states to which the respective candidate states are allowed to change, and of the candidate states to which the preceding state of the signal light is allowed to change, the processor identifies the state of the signal light as a candidate state having a maximum confidence score;
• and
• wherein the candidate states include a turn-on state in which the signal light is turned on, a turn-off state in which the signal light is turned off, an ON state during which the signal light is on, and an OFF state during which the signal light is off.