Methods for performing self-supervised learning of deep-learning based detection network by using deep Q-network and devices using the same

A method for performing self-supervised learning of a deep-learning based detection network by using a deep Q-network, comprising steps of:

• (a) on condition that (i) a detection network capable of performing an object detection has been trained with at least one training data stored in a training database and then a learning device, upon acquiring at least one first unlabeled image from an unlabeled database, has performed or supported another device to perform processes of (i-1) inputting the first unlabeled image into the detection network, to thereby allow the detection network to perform the object detection on the first unlabeled image and thus to generate first object detection information, (i-2) generating a first state set by referring to the first object detection information, wherein the first state set includes a (1_1)-st classification confidence to a (1_k1)-th classification confidence, a (1_1)-st regression uncertainty to a (1_k1)-th regression uncertainty, and a (1_1)-st pooled feature map to a (1_k1)-th pooled feature map, respectively corresponding to a (1_1)-st bounding box indicating a 1st object included in the first unlabeled image to a (1_k1)-th bounding box indicating a k1th object included in the first unlabeled image, and wherein k1 is an integer bigger than or equal to 1, (i-3) inputting the first state set into a deep Q-network, to thereby allow the deep Q-network to perform a learning operation on the first state set and thus to generate a Q-value including an acceptance score and a rejection score, and (i-4) generating an action by applying an argMax function to the Q-value, (ii) (ii-1) (1) if the action is determined as corresponding to an acceptance of the first unlabeled image, adding a labeled image to the training database as the training data wherein the labeled image is generated by inserting the first object detection information into the first unlabeled image, deleting the first unlabeled image from the unlabeled database, retraining the detection network by using at least one of the training data included in the training database, and thus generating a reward by referring to a first accuracy wherein the first accuracy is obtained from a result of testing the detection network by using a validation database, (2) if the action is determined as corresponding to a rejection of the first unlabeled image, deleting the first unlabeled image from the unlabeled database and generating the reward by referring to a second accuracy wherein the second accuracy is obtained from a result of testing the detection network by using the validation database, and (ii-2) upon acquiring at least one second unlabeled image from the unlabeled database, (1) inputting the second unlabeled image into the detection network, to thereby allow the detection network to perform the object detection on the second unlabeled image and thus to generate second object detection information, (2) generating a second state set by referring to the second object detection information, wherein the second state set includes a (2_1)-st classification confidence to a (2_k2)-th classification confidence, a (2_1)-st regression uncertainty to a (2_k2)-th regression uncertainty, and a (2_1)-st pooled feature map to a (2_k2)-th pooled feature map, respectively corresponding to a (2_1)-st bounding box indicating a 1st object included in the second unlabeled image to a (2_k2)-th bounding box indicating a k2th object included in the second unlabeled image, and wherein k2 refers to an integer bigger than or equal to 1, and (3) storing the first state set, the action, the reward, and the second state set as elements of a transition vector in a memory, and (iii) generating a minibatch by sampling at least one specific transition vector to be used as at least one transition vector for training from the memory and then training the deep Q-network by using the minibatch such that the deep Q-network outputs the action for increasing the reward according to the first state set and the second state set, a self-learning device, upon acquiring at least one new unlabeled image from a new unlabeled database, performing or supporting another device to perform processes of (i) inputting the new unlabeled image into the detection network, to thereby allow the detection network to perform the object detection on the new unlabeled image and thus to generate new object detection information, (ii) generating a new state set by referring to the new object detection information, wherein the new state set includes a first new classification confidence to a k3th new classification confidence, a first new regression uncertainty to a k3th new regression uncertainty, and a first new pooled feature map to a k3th new pooled feature map, respectively corresponding to a first new bounding box indicating a 1st object included in the new unlabeled image to a k3th new bounding box indicating a k3th object included in the new unlabeled image, and wherein the k3 refers to an integer bigger than or equal to 1, (iii) inputting the new state set into the deep Q-network, to thereby allow the deep Q-network to perform the learning operation on the new state set and thus to generate a new Q-value including a new acceptance score and a new rejection score, and (iv) generating a new action by applying the argMax function to the new Q-value;
• and
• (b) the self-learning device performing or supporting another device to perform processes of (i) if the new action is determined as corresponding to an acceptance of the new unlabeled image, adding a new labeled image to the training database as the training data wherein the new labeled image is generated by inserting the new object detection information into the new unlabeled image, deleting the new unlabeled image from the new unlabeled database, and retraining the detection network by using at least one of the training data included in the training database, and (ii) if the new action is determined as corresponding to a rejection of the new unlabeled image, deleting the new unlabeled image from the new unlabeled database.

A learning device for performing self-supervised learning of a deep-learning based detection network by using a deep Q-network, comprising:

• at least one memory that stores instructions;
• and
• at least one processor configured to execute the instructions to perform or support another device to perform:
• processes of (I) on condition that a detection network capable of performing an object detection has been trained with at least one training data stored in a training database, upon acquiring at least one first unlabeled image from an unlabeled database, (i) inputting the first unlabeled image into the detection network, to thereby allow the detection network to perform the object detection on the first unlabeled image and thus to generate first object detection information, (ii) generating a first state set by referring to the first object detection information, wherein the first state set includes a (1_1)-st classification confidence to a (1_k1)-th classification confidence, a (1_1)-st regression uncertainty to a (1_k1)-th regression uncertainty, and a (1_1)-st pooled feature map to a (1_k1)-th pooled feature map, respectively corresponding to a (1_1)-st bounding box indicating a 1st object included in the first unlabeled image to a (1_k1)-th bounding box indicating a k1th object included in the first unlabeled image, and wherein k1 is an integer bigger than or equal to 1, (iii) inputting the first state set into a deep Q-network, to thereby allow the deep Q-network to perform a learning operation on the first state set and thus to generate a Q-value including an acceptance score and a rejection score, and (iv) generating an action by applying an argMax function to the Q-value, (II) (i) (i-1) if the action is determined as corresponding to an acceptance of the first unlabeled image, adding a labeled image to the training database as the training data wherein the labeled image is generated by inserting the first object detection information into the first unlabeled image, deleting the first unlabeled image from the unlabeled database, retraining the detection network by using at least one of the training data included in the training database, and thus generating a reward by referring to a first accuracy wherein the first accuracy is obtained from a result of testing the detection network by using a validation database, (i-2) if the action is determined as corresponding to a rejection of the first unlabeled image, deleting the first unlabeled image from the unlabeled database and generating the reward by referring to a second accuracy wherein the second accuracy is obtained from a result of testing the detection network by using the validation database, and (ii) upon acquiring at least one second unlabeled image from the unlabeled database, (ii-1) inputting the second unlabeled image into the detection network, to thereby allow the detection network to perform the object detection on the second unlabeled image and thus to generate second object detection information, (ii-2) generating a second state set by referring to the second object detection information, wherein the second state set includes a (2_1)-st classification confidence to a (2_k2)-th classification confidence, a (2_1)-st regression uncertainty to a (2_k2)-th regression uncertainty, and a (2_1)-st pooled feature map to a (2_k2)-th pooled feature map, respectively corresponding to a (2_1)-st bounding box indicating a 1st object included in the second unlabeled image to a (2_k2)-th bounding box indicating a k2th object included in the second unlabeled image, and wherein k2 refers to an integer bigger than or equal to 1, and (ii-3) storing the first state set, the action, the reward, and the second state set as elements of a transition vector in a memory, and (III) generating a minibatch by sampling at least one specific transition vector to be used as at least one transition vector for training from the memory and then training the deep Q-network by using the minibatch such that the deep Q-network outputs the action for increasing the reward according to the first state set and the second state set.

A self-learning device for performing self-supervised learning of a deep-learning based detection network by using a deep Q-network, comprising:

• at least one memory that stores instructions;
• and
• at least one processor configured to execute the instructions to perform or support another device to perform:
• processes of (I) on condition that (i) a detection network capable of performing an object detection has been trained with at least one training data stored in a training database and then a learning device, upon acquiring at least one first unlabeled image from an unlabeled database, has performed or supported another device to perform processes of (i-1) inputting the first unlabeled image into the detection network, to thereby allow the detection network to perform the object detection on the first unlabeled image and thus to generate first object detection information, (i-2) generating a first state set by referring to the first object detection information, wherein the first state set includes a (1_1)-st classification confidence to a (1_k1)-th classification confidence, a (1_1)-st regression uncertainty to a (1_k1)-th regression uncertainty, and a (1_1)-st pooled feature map to a (1_k1)-th pooled feature map, respectively corresponding to a (1_1)-st bounding box indicating a 1st object included in the first unlabeled image to a (1_k1)-th bounding box indicating a k1th object included in the first unlabeled image, and wherein k1 is an integer bigger than or equal to 1, (i-3) inputting the first state set into a deep Q-network, to thereby allow the deep Q-network to perform a learning operation on the first state set and thus to generate a Q-value including an acceptance score and a rejection score, and (i-4) generating an action by applying an argMax function to the Q-value, (ii) (ii-1) (1) if the action is determined as corresponding to an acceptance of the first unlabeled image, adding a labeled image to the training database as the training data wherein the labeled image is generated by inserting the first object detection information into the first unlabeled image, deleting the first unlabeled image from the unlabeled database, retraining the detection network by using at least one of the training data included in the training database, and thus generating a reward by referring to a first accuracy wherein the first accuracy is obtained from a result of testing the detection network by using a validation database, (2) if the action is determined as corresponding to a rejection of the first unlabeled image, deleting the first unlabeled image from the unlabeled database and generating the reward by referring to a second accuracy wherein the second accuracy is obtained from a result of testing the detection network by using the validation database, and (ii-2) upon acquiring at least one second unlabeled image from the unlabeled database, (1) inputting the second unlabeled image into the detection network, to thereby allow the detection network to perform the object detection on the second unlabeled image and thus to generate second object detection information, (2) generating a second state set by referring to the second object detection information, wherein the second state set includes a (2_1)-st classification confidence to a (2_k2)-th classification confidence, a (2_1)-st regression uncertainty to a (2_k2)-th regression uncertainty, and a (2_1)-st pooled feature map to a (2_k2)-th pooled feature map, respectively corresponding to a (2_1)-st bounding box indicating a 1st object included in the second unlabeled image to a (2_k2)-th bounding box indicating a k2th object included in the second unlabeled image, and wherein k2 refers to an integer bigger than or equal to 1, and (3) storing the first state set, the action, the reward, and the second state set as elements of a transition vector in a memory, and (iii) generating a minibatch by sampling at least one specific transition vector to be used as at least one transition vector for training from the memory and then training the deep Q-network by using the minibatch such that the deep Q-network outputs the action for increasing the reward according to the first state set and the second state set, upon acquiring at least one new unlabeled image from a new unlabeled database, (i) inputting the new unlabeled image into the detection network, to thereby allow the detection network to perform the object detection on the new unlabeled image and thus to generate new object detection information, (ii) generating a new state set by referring to the new object detection information, wherein the new state set includes a first new classification confidence to a k3th new classification confidence, a first new regression uncertainty to a k3th new regression uncertainty, and a first new pooled feature map to a k3th new pooled feature map, respectively corresponding to a first new bounding box indicating a 1st object included in the new unlabeled image to a k3th new bounding box indicating a k3th object included in the new unlabeled image, and wherein the k3 refers to an integer bigger than or equal to 1, (iii) inputting the new state set into the deep Q-network, to thereby allow the deep Q-network to perform the learning operation on the new state set and thus to generate a new Q-value including a new acceptance score and a new rejection score, and (iv) generating a new action by applying the argMax function to the new Q-value, and (II) (i) if the new action is determined as corresponding to an acceptance of the new unlabeled image, adding a new labeled image to the training database as the training data wherein the new labeled image is generated by inserting the new object detection information into the new unlabeled image, deleting the new unlabeled image from the new unlabeled database, and retraining the detection network by using at least one of the training data included in the training database, and (ii) if the new action is determined as corresponding to a rejection of the new unlabeled image, deleting the new unlabeled image from the new unlabeled database.