A method of training a transcoder algorithm having a parameter space using a training dataset comprising audio samples having high resource and low resource language components, the method comprising:
- by one or more computing devices:
- evaluating batches of audio samples from a training dataset based on at least one language component selected from a group consisting of accent, dialect, accented and language specific domain, accented and language specific subdomain, dialect and language specific domain, dialect and language specific subdomain;
- creating, using at least two audio samples from the batches of audio samples, one or more additional audio samples;
- adding the one or more additional audio samples to the batches of audio samples;
- segmenting, the batches of audio samples, into one or more minibatches by sampling the batches of audio samples;
- generating, using a pre-trained neural network, one or more encodings comprising a hidden representation for each audio sample of the one or more minibatches;
- generating, using an accent predictor model, an accent prediction for each audio sample of the one or more minibatches;
- generating, using a domain predictor model, a domain prediction for each audio sample of the one or more minibatches;
- dynamically combining, using a learnable accent adapter implemented as an attention layer, the hidden representation, the accent prediction, and the domain prediction to produce a combined output, wherein the attention layer takes the hidden representation as a query and the accent prediction and the domain prediction as keys and computes attention scores;
- generating, using the combined output, one or more transcript predictions for each audio sample of the one or more minibatches;
- comparing the one or more transcript predictions for each audio sample of the one or more minibatches to one or more true transcript labels for each audio sample of the one or more minibatches to determine a transcript loss for each audio sample of the one or more minibatches;
- comparing the accent prediction for each audio sample of the one or more minibatches to a true accent label for each audio sample of the one or more minibatches to determine an accent loss for each audio sample of the one or more minibatches;
- comparing the domain prediction for each audio sample of the one or more minibatches to a true domain label for each audio sample of the one or more minibatches to determine a domain loss for each audio sample of the one or more minibatches;
- aggregating, for each audio sample of the one or more minibatches, the transcript loss, the accent loss, and the domain loss as a weighted sum to form an aggregated loss score;
- and
- tuning the parameter space of the transcoder algorithm using the aggregated loss score, wherein the generating steps, the comparing steps, and the aggregating step are performed iteratively using the one or more minibatches.