Back to results
US 12,762,466 B2Granted
Learning apparatus and learning method
AssigneeDENSO TEN Limited
InventorsRyusuke Seki, Yasutaka Okada, Yuki Katayama, Rei Hiromi, Aoi Ogishima
Life of this patent
Estimating the term…
Abstract
A learning apparatus for training a student model with a teacher model includes a processor. The processor computes the performance difference between the teacher and student models. The processor makes at least one of a determination, based on the performance difference, of whether to use the teacher model and a determination, based on the performance difference, of whether to change the weight coefficient in calculating the loss in the student model.
Claims7
A learning method performed by a processor to perform training of a student model with a teacher model, comprising:
- continuously training the student model by:
- during the training, determining a loss to update a parameter of the student model, the loss including a loss in an inference result based on the teacher model with respect to learning data that is image data, the loss in the inference result including recognition of the type and position of an object, and the position is identified with a BBOX (bounding box);
- after the training of the student model with the teacher model, determining a performance difference between the teacher model and the student model;
- and
- when the performance difference becomes smaller than a predetermined threshold value, changing the teacher model used in training the student model to a teacher model with higher performance than the teacher model currently used and then training the student model.
2 more elements
A learning method performed by a processor to perform training of a student model with a teacher model, comprising:
- continuously training the student model by:
- during the training, determining a loss in the student model according to expression (A) below to update a parameter of the student model;
- after the training, comparing performance of the student model with performance of the teacher model;
- and
- performing the training while decreasing a value of 2 in expression (A) as the performance of the student model approaches the performance of the teacher model,
- L
- =
- L
-
- e
-
- q
- +
- λ
- ·
- Ldstl
- (
- A
- )
- where
- L represents the loss in the student model;
- Leq represents a loss from learning data;
- Ldstl represents a loss from the teacher model;
- and
- λ represents a weight coefficient for adjusting a balance between the loss from learning data and the loss from the teacher model,
- wherein the learning data is image data and the student model and the teacher model are models for object detection that include identifications of types and locations of objects,
- wherein a loss in an inference result based on the teacher model with respect to the learning data includes recognition of the type and position of an object, and the position is identified with a BBOX (bounding box).
24 more elements
A learning method performed by a processor to perform training of a student model with a teacher model, comprising:
- continuously training the student model by:
- computing:
- a first loss, which is a loss in an inference result based on the teacher model with respect to learning data, and
- a second loss, which is a loss in an inference result based on the student model with respect to the learning data;
- if the first loss is smaller than the second loss, computing a loss in the student model by using the second loss and a loss from the teacher model and updating a parameter of the student model based on the computed loss;
- and
- if the first loss is equal to or larger than the second loss, updating the parameter of the student model with the second loss taken as a loss in the student model,
- wherein the learning data is image data and the student model and the teacher model are models for object detection that include identifications of types and locations of objects, and
- wherein a loss in an inference result based on the teacher model with respect to the learning data includes recognition of the type and position of an object, and the position is identified with a BBOX (bounding box).
6 more elements
A learning method performed by a processor to perform training of a student model with a teacher model, comprising:
- continuously training the student model by:
- computing:
- a first loss, which is a loss in an inference result based on the teacher model with respect to learning data, and
- a second loss, which is a loss in an inference result based on the student model with respect to the learning data,
- if the first loss is smaller than the second loss and a difference between the first and second losses is equal to or larger than a predetermined threshold value, to determine to perform training using the teacher model, and
- if the first loss is smaller than the second loss and the difference between the first and second losses is less than the predetermined threshold value, or if the first loss is equal to or larger than the second loss, changing the teacher model or changing a weight coefficient for adjusting a balance between a loss from learning data and a loss from the teacher model,
- wherein the learning data is image data and the student model and the teacher model are models for object detection that include identifications of types and locations of objects, and
- wherein a loss in an inference result based on the teacher model with respect to the learning data includes recognition of the type and position of an object, and the position is identified with a BBOX (bounding box).
5 more elements
Dependencies are inferred from the claim text.