# US 12,763,067 B2 · System and method for encoding and transcoding audio data comprising language, accent, domain specific components

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,763,067 B2
- Document ID: US12763067B2
- Status: Granted
- Kind: B2
- Application number: 18/355,166
- Filed: 2023-07-19
- Published: 2025-01-23
- Granted: 2026-10-13
- Earliest claimed: 2022-05-19
- Applicants: Oloruntobiloba Tolulope Olatunji
- Inventors: Oloruntobiloba Tolulope Olatunji
- Web page: https://patentgrep.com/patent/US12763067B2

## CPC

- G10L 15/063: {Training}
- G10L 15/16: using artificial neural networks
- G10L 2015/0631: {Creating reference templates; Clustering}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

A method of training a transcoder algorithm using audio samples having high resource and low resource language components. The method includes evaluating batches of audio samples based on accent, dialect, accented and language specific domain, accented and language specific subdomain, dialect and language specific domain, dialect and language specific subdomain, or combinations thereof. The method includes up-sampling audio samples having at least one language component from the group and adding the up-sampled audio samples to the batches of audio samples. The method includes generating hidden representations using a pre-trained neural network, measuring differences of delta between predictions and true labels from metadata and predicting a score for each task. The method also includes comparing the predicted score to a true label from the metadata of each audio and tuning the parameter space of the transcoder algorithm based on the comparison between the predicted score and the known score.

## Claims

### Claim 1

A method of training a transcoder algorithm having a parameter space using a training dataset comprising audio samples having high resource and low resource language components, the method comprising:
by one or more computing devices:
evaluating batches of audio samples from a training dataset based on at least one language component selected from a group consisting of accent, dialect, accented and language specific domain, accented and language specific subdomain, dialect and language specific domain, dialect and language specific subdomain;
creating, using at least two audio samples from the batches of audio samples, one or more additional audio samples;
adding the one or more additional audio samples to the batches of audio samples;
segmenting, the batches of audio samples, into one or more minibatches by sampling the batches of audio samples;
generating, using a pre-trained neural network, one or more encodings comprising a hidden representation for each audio sample of the one or more minibatches;
generating, using an accent predictor model, an accent prediction for each audio sample of the one or more minibatches;
generating, using a domain predictor model, a domain prediction for each audio sample of the one or more minibatches;
dynamically combining, using a learnable accent adapter implemented as an attention layer, the hidden representation, the accent prediction, and the domain prediction to produce a combined output, wherein the attention layer takes the hidden representation as a query and the accent prediction and the domain prediction as keys and computes attention scores;
generating, using the combined output, one or more transcript predictions for each audio sample of the one or more minibatches;
comparing the one or more transcript predictions for each audio sample of the one or more minibatches to one or more true transcript labels for each audio sample of the one or more minibatches to determine a transcript loss for each audio sample of the one or more minibatches;
comparing the accent prediction for each audio sample of the one or more minibatches to a true accent label for each audio sample of the one or more minibatches to determine an accent loss for each audio sample of the one or more minibatches;
comparing the domain prediction for each audio sample of the one or more minibatches to a true domain label for each audio sample of the one or more minibatches to determine a domain loss for each audio sample of the one or more minibatches;
aggregating, for each audio sample of the one or more minibatches, the transcript loss, the accent loss, and the domain loss as a weighted sum to form an aggregated loss score; and
tuning the parameter space of the transcoder algorithm using the aggregated loss score, wherein the generating steps, the comparing steps, and the aggregating step are performed iteratively using the one or more minibatches.

### Claim 2 (depends on 1)

The method of claim 1, further comprises preprocessing the batches of audio samples performing at least one selected from a group consisting of:
overlaying audio sample with a white noise; randomly perturbing volume of audio samples; randomly mask short segments of audio samples; randomly fade in and fade out a percentage of audio samples at a predetermined period; randomly shift start and end positions of a percentage of samples; randomly perturb pitch of 10% of audio samples; and randomly speed up or slow down clip speed for a percentage of audio samples.

### Claim 3 (depends on 1)

The method of claim 1, wherein creating the one or more additional audio samples further comprises:
concatenating the at least two audio samples from the batches of audio samples into a single new audio waveform, each of the at least two audio samples having a same at least one language component, wherein the single new audio waveform is distinct from a minibatch, a cluster, and a resampled copy of an existing audio sample; or
concatenating the at least two audio samples from the batches of audio samples into a single new audio waveform, each of the at least two audio samples having a different at least one language component, wherein the single new audio waveform is distinct from a minibatch, a cluster, and a resampled copy of an existing audio sample.

### Claim 4 (depends on 1)

The method of claim 1, wherein the learnable accent adapter dynamically selects, based on the attention scores, sub-components of the accent prediction and the domain prediction that are valuable for generating the transcript predictions.

### Claim 5 (depends on 1)

The method of claim 1, further comprising, for each audio sample of the one or more minibatches, weighting the transcript loss, the accent loss, and the domain loss prior to forming the aggregated loss score.

### Claim 6 (depends on 1)

The method of claim 1, wherein the segmenting comprises composing minibatches for each training epoch such that majority accents make up a first percentage of the minibatches in earlier epochs and majority accents and minority accents each make up a substantially equal percentage of the minibatches in later epochs.