Alternative soft label generation
AssigneeInternational Business Machines Corporation
InventorsToru Nagano, Takashi Fukuda, Gakuto Kurata
Life of this patent
Estimating the term…
Abstract
An approach to identifying alternate soft labels for training a student model may be provided. A teaching model may generate a soft label for a labeled training data. The training data can be an acoustic file for speech or a spoken natural language. A pool of soft labels previously generated by teacher models can be searched at the label level to identify soft labels that are similar to the generated soft label. The similar soft labels can have similar length or sequence at the word phoneme, and/or state level. The identified similar soft labels can be used in conjunction with the generated soft label to train a student model.
Claims20
A computer-implemented method for training a student model via knowledge distillation with supplemented labeled training data, the method comprising:
- receiving, by one or more processors, a labeled acoustic feature;
- generating, by the one or more processors, a soft label for the labeled acoustic feature by a teacher model;
- extracting, by the one or more processors, a second soft label similar to the first soft label from a soft label pool;
- and
- training, by the one or more processors, a student model using the first soft label and second soft label.
A system for generating labeled training data for a student model, the system comprising:
- one or more computer processors;
- one or more computer readable storage media;
- and
- computer program instructions to:
- receive a first training data including a plurality of pairs of a first hard label and a first soft label, wherein each first soft label is generated by a teacher model;
- detecting, by the one or more processors, a second soft label similar to the first soft label from a soft labels pool;
- output the second soft label as an additional training data for each first soft label;
- and
- train a student model using the first soft label and second soft label.
A computer program product for code refactor renaming comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processors to perform a function, the function comprising:
- receive a first training data including a plurality of pairs of a first hard label and a first soft label, wherein each first soft label is generated by a teacher model;
- detecting, by the one or more processors, a second soft label similar to the first soft label from a soft labels pool;
- output the second soft label as an additional training data for each first soft label;
- and
- train a student model using the first soft label and second soft label.
Dependencies are inferred from the claim text.