Method and device for providing information for evaluating driving habits of driver by detecting driving scenarios occurring during driving

A computer-implemented learning method for detecting driving events occurring during driving, thereby to detect driving scenarios including at least part of the driving events, comprising steps of:

• instructing a generating network included in a generative adversarial network (GAN) to:
• select a plurality of driving events in a specific order,
• generate an event vector by using pieces of information on each of the driving events, the event vector enumerating the selected driving events in the specific order and including the pieces of the information as its components;
• and
• generate a ground-truth (GT) driving scenario corresponding to the driving events in the specific order;
• delivering the event vector and the GT driving scenario to a recurrent neural network (RNN) as training data responsive to the event vector and the GT driving scenario satisfying a prescribed verification rule, wherein the verification rule corresponds to one or more road rules which have been violated by drivers for a prescribed number of times during a time range of the past, and wherein the event vector and the GT driving scenario satisfy the verification rule when one or more circumstances of complying with or violating the road rules are reflected in the event vector and the GT driving scenario;
• upon the RNN acquiring the event vector, applying, by the RNN, one or more RNN operations to the components of the event vector, in the specific order, thereby to detect at least one predicted driving scenario including at least part of the driving events;
• and
• generating, by the RNN, an RNN loss by referring to the specific predicted driving scenario and the GT driving scenario and to perform a backpropagation through time (BPTT) by using the RNN loss, thereby to learn at least part of a plurality of parameters of the RNN.

A testing method for detecting driving scenarios occurring during driving, thereby to evaluate driving habits of a subject driver, comprising steps of:

• learning a part of a plurality of parameters of a recurrent neural network (RNN) by:
• acquiring at least one event vector and a ground-truth (GT) driving scenario for training,
• wherein the event vector includes as its components a plurality of pieces of information on each of a plurality of driving events in a specific order,
• wherein the event vector and the GT driving scenario are acquired by the RNN responsive to the event vector and the GT driving scenario satisfying a prescribed verification rule,
• wherein the verification rule corresponds to one or more road rules that have been violated by drivers for a prescribed number of times during a time range of the past, and
• wherein the event vector and the GT driving scenario satisfy the verification rule when one or more circumstances of complying with or violating the road rules are reflected in the event vector and the GT driving scenario;
• in response to the acquisition of the at least one event vector for training, instructing the RNN to apply one or more RNN operations to the components of the event vector in the specific order, thereby to detect at least one predicted driving scenario for training including at least part of the driving events for training;
• generating an RNN loss by referring to the predicted driving scenario and the GT driving scenario and performing a backpropagation through time (BPTT) by using the RNN loss;
• detecting a plurality of driving events for testing,
• wherein the driving events for testing comprise interior operation information and exterior circumstance information, the interior operation information including direction information, velocity information, turn signal information, and emergency signal information, and the exterior circumstance information including information about a traffic sign located closer than a first threshold from a subject vehicle, surrounding motion information of surrounding vehicles located closer than a second threshold from the subject vehicle, and lane information about a lane that includes the subject vehicle, and
• wherein the driving events for testing are captured by at least one of an electrical control unit (ECU) and an external circumstance monitor, the external circumstance monitor comprising an external camera and a V2X communicator;
• in response to one or more of the driving events for testing being detected, instructing the RNN to apply the RNN operations to each of the driving events for testing in sequence, thereby to detect one or more predicted driving scenarios for testing including at least part of the driving events for testing;
• and
• generating evaluation information to be used for evaluating the driving habits of the subject driver of the subject vehicle, by referring to at least one of the predicted driving scenarios for testing.

A learning device for detecting driving events occurring during driving, thereby to detect driving scenarios including at least part of the driving events, comprising:

• at least one memory that stores instructions;
• and
• at least one processor configured to execute the instructions to perform processes of:
• acquiring an event vector and a ground-truth (GT) driving scenario by performing processes of:
• providing a plurality of graphical objects corresponding to a plurality of driving events through a display, each of the graphical objects corresponding to each of the driving events respectively, each of the graphical objects being individually selectable by a user;
• in response to the plurality of the graphical objects corresponding to the driving events being selected by the user in a specific order, generating the event vector by using pieces of information on said each of the selected driving events, the event vector enumerating the selected driving events in the specific order and including the pieces of the information as its components;
• generating the GT driving scenario corresponding to the driving events in the specific order,
• delivering the event vector and the GT driving scenario to a recurrent neural network (RNN) as training data responsive to the event vector and the GT driving scenario satisfying a prescribed verification rule, wherein the verification rule corresponds to one or more road rules which have been violated by drivers for a prescribed number of times during a time range of the past, and wherein the event vector and the GT driving scenario satisfy the verification rule when one or more circumstances of complying with or violating the road rules are reflected in the event vector and the GT driving scenario;
• upon acquiring the event vector, instructing the RNN to apply one or more RNN operations to the components of the event vector, in the specific order, thereby to detect at least one predicted driving scenario including at least part of the driving events;
• and
• generating an RNN loss by referring to the predicted driving scenario and the GT driving scenario and performing a backpropagation through time (BPTT) by using the RNN loss, thereby to learn at least part of a plurality of parameters of the RNN.

A learning device for detecting driving events occurring during driving, thereby to detect driving scenarios including at least part of the driving events, comprising:

• at least one memory that stores instructions;
• and
• at least one processor configured to execute the instructions to perform processes of:
• instructing a generating network included in a generative adversarial network (GAN) to:
• select a plurality of driving events in a specific order,
• generate an event vector by using pieces of information on each of the driving events, the event vector enumerating the selected driving events in the specific order and including the pieces of the information as its components;
• and
• generate a ground-truth (GT) driving scenario corresponding to the driving events in the specific order,
• delivering the event vector and the GT driving scenario to a recurrent neural network (RNN) as training data responsive to the event vector and the GT driving scenario satisfying a prescribed verification rule, wherein the verification rule corresponds to one or more road rules which have been violated by drivers for a prescribed number of times during a time range of the past, and wherein the event vector and the GT driving scenario satisfy the verification rule when one or more circumstances of complying with or violating the road rules are reflected in the event vector and the GT driving scenario;
• upon acquiring the event vector, instructing the RNN to apply one or more RNN operations to the components of the event vector, in the specific order, thereby to detect at least one predicted driving scenario including at least part of the driving events;
• and
• generating an RNN loss by referring to the predicted driving scenario and the GT driving scenario and performing a backpropagation through time (BPTT) by using the RNN loss, thereby to learn at least part of a plurality of parameters of the RNN.

A testing device for detecting driving scenarios occurring during driving, thereby to evaluate driving habits of a subject driver, comprising:

• at least one memory that stores instructions;
• and
• at least one processor configured to execute the instructions to perform processes of:
• learning a part of a plurality of parameters of a recurrent neural network (RNN) by:
• acquiring at least one event vector and a ground-truth (GT) driving scenario for training,
• wherein the event vector includes as its components a plurality of pieces of information on each of a plurality of driving events in a specific order,
• wherein the event vector and the GT driving scenario are acquired by the RNN responsive to the event vector and the GT driving scenario satisfying a prescribed verification rule,
• wherein the verification rule corresponds to one or more road rules that have been violated by drivers for a prescribed number of times during a time range of the past, and
• wherein the event vector and the GT driving scenario satisfy the verification rule when one or more circumstances of complying with or violating the road rules are reflected in the event vector and the GT driving scenario;
• in response to the acquisition of the at least one event vector for training, instructing a recurrent neural network (RNN) to apply one or more RNN operations to the components for training, of the event vector, in the specific order, thereby to detect at least one predicted driving scenario for training including at least part of the driving events for training;
• generating an RNN loss by referring to the predicted driving scenario for training and the GT driving scenario, and performing a backpropagation through time (BPTT) by using the RNN loss;
• detecting a plurality of driving events for testing,
• wherein the driving events for testing comprise interior operation information and exterior circumstance information, the interior operation information including direction information, velocity information, turn signal information, and emergency signal information, and the exterior circumstance information including information about a traffic sign located closer than a first threshold from a subject vehicle, surrounding motion information of surrounding vehicles located closer than a second threshold from the subject vehicle, and lane information about a lane that includes the subject vehicle, and
• wherein the driving events for testing are captured by at least one of an electrical control unit (ECU) and an external circumstance monitor, the external circumstance monitor comprising an external camera and a V2X communicator;
• in response to one or more of the driving events for testing being detected, instructing the RNN to apply the RNN operations to each of the driving events for testing in sequence, thereby to detect one or more predicted driving scenarios for testing including at least part of the driving events for testing;
• and
• generating evaluation information to be used for evaluating the driving habits of the subject driver of a subject vehicle, by referring to at least one of the predicted driving scenarios for testing.