Apparatus and method for controlling traffic signals of traffic lights in sub-area by using reinforcement learning model

An apparatus for controlling traffic signals of a plurality of traffic lights in a sub-area by using a neural network model, the apparatus comprising:

• a memory storing at least one program;
• and
• at least one processor configured to drive the neural network model by executing the at least one program,
• wherein the at least one processor is further configured to
• configure state information of the sub-area by using downstream information for each of a plurality of intersections included in the sub-area, the downstream information being obtained in a current cycle time,
• input the state information into trained neural network model and obtain an action of the sub-area by using an output of the trained neural network model, the action including green times and offsets,
• determine whether an offset is set to within a preset absolute value range;
• and
• generate a coordinated signal value for applying action information to the plurality of traffic lights in the sub-area during a transition process configured with a plurality of subsequent cycle times, in response to determining that the offset is set to a value out of the preset absolute value range,
• wherein the neural network model is trained using a reinforcement learning algorithm which is based on the action information, the state information, and reward information, and
• wherein the reward information for each intersection is defined as arithmetic mean of stop rates obtained at downstream of each of a plurality of links of the intersection, each of the stop rates is a value obtained by dividing a processed queue length by a processed traffic volume of the downstream of one of the plurality links of the intersection.

A non-transitory computer-readable recording medium that stores a program that, when executed by a computer, configures the computer to:

• configure state information of a sub-area by using downstream information for a current cycle time, wherein the downstream information is configured for each of a plurality of intersections included in the sub-area,
• obtain action information including green times and offsets for the sub-area by inputting the state information to a trained neural network model,
• determine whether an offset is within a preset absolute value range;
• and
• generate a coordinated signal value for applying the action information to a plurality of traffic lights in the sub-area during a transition process configured with a plurality of subsequent cycle times, in response to determining that the offset is set to a value out of the preset absolute value range,
• wherein the neural network model is trained using a reinforcement learning algorithm which is based on the action information, the state information, and reward information, and
• wherein the reward information for each intersection is defined as arithmetic mean of stop rates obtained at downstream of each of a plurality of links of the intersection, each of the stop rates is a value obtained by dividing a processed queue length by a processed traffic volume of the downstream of one of the plurality links of the intersection.