Knowledge Distillation and Gradient Pruning-Based Compression of Artificial Intelligence-Based Base Caller
AssigneeIllumina, Inc.
InventorsAnindita DUTTA, Gery VESSERE, Dorna KASHEFHAGHIGHI, Kishore JAGANATHAN, Amirali KIA
Life of this patent
Estimating the term…
Abstract
The technology disclosed compresses a larger, teacher base caller into a smaller, student base caller. The student base caller has fewer processing modules and parameters than the teacher base caller. The teacher base caller is trained using hard labels (e.g., one-hot encodings). The trained teacher base caller is used to generate soft labels as output probabilities during the inference phase. The soft labels are used to train the student base caller.
Claims20
An artificial intelligence-based method of base calling, the method including:
- training a first base caller by using a first set of cluster images as training data,
- wherein the first set of cluster images are annotated with first ground truth data that uses discrete valued labels to identify a correct base call;
- evaluating a second set of cluster images as inference data by applying the trained first base caller on the second set of cluster images and generating base call predictions,
- wherein the base call predictions are represented by continuous valued weights that identify a predicted base call;
- training a second base caller using the second set of cluster images as training data,
- wherein the second set of cluster images are annotated with second ground truth data that identifies a correct base call based on
- (i) the discrete valued labels, and
- (ii) the continuous valued weights;
- wherein the second base caller has fewer processing modules and parameters than the first base caller;
- and
- evaluating a third set of cluster images as inference data by applying the trained second base caller on the third set of cluster images and generating base call predictions.
A system for artificial intelligence-based base calling, comprising:
- a base caller trained on cluster images that are annotated with ground truth data that identifies a correct base call based on
- (i) discrete valued labels of ground truth data used to train another base caller, and
- (ii) continuous valued weights of base call predictions generated by the another base caller for the cluster images during inference;
- wherein the base caller has fewer processing modules and parameters than the another base caller;
- and
- wherein the base caller is configured to evaluate additional cluster images and generate, for the additional cluster images, base call predictions.
A system for artificial intelligence-based base calling, comprising:
- a first base caller trained on cluster images that are annotated with ground truth data that identifies a correct base call based on base call predictions generated by a second base caller.
Dependencies are inferred from the claim text.