# US 2024/0406314 A1 · Performance optimization for real-time large language speech to text systems

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0406314 A1
- Document ID: US20240406314A1
- Status: Published application
- Kind: A1
- Application number: 18/204,981
- Filed: 2023-06-02
- Published: 2024-12-05
- Earliest claimed: 2023-06-02
- Assignees: Bank of America Corporation
- Applicants: Bank of America Corporation
- Inventors: Emad Noorizadeh; Emmanuel Dibia; Jennifer Russell; Rajan Jhaveri
- Web page: https://patentgrep.com/patent/US20240406314A1

## CPC

- H04M 3/4936: {Speech interaction details (speech recognition per se G10L15/00)}
- G10L 15/16: using artificial neural networks
- G10L 15/26: Speech to text systems (G10L15/08 takes precedence)
- G10L 15/30: Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
- H04M: TELEPHONIC COMMUNICATION (circuits for controlling other apparatus via a telephone cable and not involving telephone switching apparatus G08)
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

Methods and systems for transcribing communications are provided. Methods may include receiving a communication. Methods may include splitting the communication into a plurality of communication segments. Each communication segment may include two or more words. Methods may include transcribing each segment included in the plurality of communication segments, in parallel. The transcribing may include using a transformer neural network to transcribe each segment included in the plurality of communication segments. Methods may include generating a transcription from the transcribing. The transcription may be generated by combining the transcription of each of the communication segments into a combined transcription. Methods may include correcting the combined transcription.

## Claims

### Claim 1

A method for transcribing communications, the method comprising:
receiving a communication;
splitting the communication into a plurality of communication segments, each communication segment comprising two or more words;
transcribing, in parallel, using a transformer neural network for each segment included in the plurality of communication segments, each segment included in the plurality of communication segments;
generating a transcription from the transcribing;
combining the transcription of each of the communication segments into a combined transcription; and
correcting the combined transcription.

### Claim 2 (depends on 1)

The method of claim 1 wherein the communication occurs between a human caller and an interactive voice response system.

### Claim 3 (depends on 1)

The method of claim 1 wherein each segment comprises thirty seconds of the communication.

### Claim 4 (depends on 1)

The method of claim 1 wherein each segment comprises a snippet of less than thirty seconds of the communication.

### Claim 5

A method for maintaining accuracy in transcribing a communication, the method comprising:
in a first environment:
receiving a communication; and
transcribing the communication, using a robust speech recognition via large-scale weak supervision model, into a first transcription;
in a second environment:
receiving the communication;
splitting the communication into a plurality of communication segments, each communication segment comprising two or more words;
identifying a number of communication segments included in the plurality of communication segments;
instantiating a plurality of instances of the robust speech recognition via large-scale weak supervision model, an integer of instances, included in the plurality of instances, being equivalent to the number of communication segments;
assigning each communication segment, included in the plurality of communication segments, to an instance of the robust speech recognition via large-scale weak supervision model, said instance being one of the plurality of instances of the robust speech recognition via large-scale weak supervision model;
transcribing, using parallel processing of the remaining instances, each communication segment, the transcribing using the assigned instance of the robust speech recognition via large-scale weak supervision model, into a transcribed communication segment;
combining the transcribed communication segments into a combined transcription; and
correcting the combined transcription using a domain-specific correction module;
in a test environment:
identifying a first quantifiable resources value consumed by transcribing the communication using the robust speech recognition via large-scale weak supervision model;
identifying a first accuracy level of the first transcription;
identifying a second quantifiable resources value consumed by transcribing the communication using the plurality of instances of the robust speech recognition via large-scale weak supervision model;
determining that the first quantifiable resources value is greater than the second quantifiable resources value by over a predetermined resources value threshold;
identifying a second accuracy level of the combined transcription;
determining that the first accuracy level is greater than the second accuracy level by over a predetermined accuracy level threshold;
identifying a third quantifiable resources value consumed by correcting the combined transcription using the domain-specific correction module;
identifying a third accuracy level of the combined transcription upon completion of correcting the combined transcription using the domain-specific correction module;
determining that the third accuracy level is equivalent to or greater than the first accuracy level;
identifying a fourth quantifiable resources value, the third quantifiable resources value including the second quantifiable resources value and the third quantifiable resources value; and
determining that the fourth quantifiable resources value is less than the first quantifiable resources value.

### Claim 6 (depends on 5)

The method of claim 5 further comprising using the second environment to transcribe incoming communications.

### Claim 7 (depends on 5)

The method of claim 5 wherein the robust speech recognition via large-scale weak supervision model comprises a transformer neural network.

### Claim 8 (depends on 5)

The method of claim 5 wherein the communication occurs between a human caller and an interactive voice response system.

### Claim 9 (depends on 5)

The method of claim 5 wherein each communication segment comprises thirty seconds of the communication.

### Claim 10 (depends on 5)

The method of claim 5 wherein each communication segment comprises less than thirty seconds of the communication.

### Claim 11

A method for maintaining accuracy in transcribing a communication, the method comprising:
in a first environment:
receiving a communication;
transcribing the communication, using a robust speech recognition via large-scale weak supervision model, into a first transcription;
identifying a first quantifiable resources value consumed by transcribing the communication using the robust speech recognition via large-scale weak supervision model; and
identifying a first accuracy level of the first transcription;
in a second environment:
receiving the communication;
splitting the communication into a plurality of communication segments, each communication segment comprising two or more words;
identifying a number of communication segments included in the plurality of communication segments;
instantiating a plurality of instances of the robust speech recognition via large-scale weak supervision model, an integer of instances, included in the plurality of instances, being equivalent to the number of communication segments;
assigning each communication segment, included in the plurality of communication segments, to an instance of the robust speech recognition via large-scale weak supervision model, said instances being one of the plurality of instances of the robust speech recognition via large-scale weak supervision model;
transcribing, using parallel processing of the remaining instances, each communication segment, the transcribing using the assigned instance of the robust speech recognition via large-scale weak supervision model, into a transcribed communication segment;
combining the transcribed communication segments into a combined transcription; and
correcting the combined transcription using a domain-specific correction module;
in a test environment:
identifying a second quantifiable resources value consumed by transcribing the communication using the plurality of instances of the robust speech recognition via large-scale weak supervision model;
guaranteeing that the first quantifiable resources value is greater than the second quantifiable resources value by over a predetermined resources value threshold;
identifying a second accuracy level of the combined transcription;
guaranteeing that the first accuracy level is greater than the second accuracy level by over a predetermined accuracy level threshold;
identifying a third quantifiable resources value consumed by correcting the combined transcription using the domain-specific correction module;
identifying a third accuracy level of the combined transcription upon completion of correcting the combined transcription using the domain-specific correction module;
guaranteeing that the third accuracy level is equivalent to or greater than the first accuracy level;
identifying a fourth quantifiable resources value, the third quantifiable resources value including the second quantifiable resources value and the third quantifiable resources value; and
guaranteeing that the fourth quantifiable resources value is less than the first quantifiable resources value.

### Claim 12 (depends on 11)

The method of claim 11 further comprising using the second environment to transcribe incoming communications.

### Claim 13 (depends on 11)

The method of claim 11 wherein the robust speech recognition via large-scale weak supervision model comprises a transformer neural network.

### Claim 14 (depends on 11)

The method of claim 11 wherein the communication occurs between a human caller and an interactive voice response system.

### Claim 15 (depends on 11)

The method of claim 11 wherein each communication segment comprises thirty seconds of the communication.

### Claim 16 (depends on 11)

The method of claim 11 wherein each communication segment comprises less than thirty seconds of the communication.

### Claim 17

A communication transcription system comprising:
a first environment, the first environment comprising:
a transformer neural network, the transformer neural network operable to:
receive an audio communication; and
transcribe the audio communication into a first transcription;
a second environment, the second environment comprising:
a receiver, the receiver operable to receive the audio communication;
a segmentation model, the segmentation model operable to segment the audio communication into a plurality of communication segments, each communication segment comprising thirty or less seconds of the audio communication;
a transcriber, the transcriber operable to:
instantiate an instance of a transformer neural network for each communication segment included in the plurality of communication segments;
transcribe, in parallel, each communication segment, included in the plurality of communication segments, using the instance of the transformer neural network instantiated for the communication segment;
combine the transcribed communication segments into a combined transcription; and
correct the combined transcription using a domain-specific corrector.

### Claim 18 (depends on 17)

The communication transcription system of claim 17 further comprising a test environment, the test environment operable to:
identify a first quantifiable resources value consumed by transcribing the communication using the transformer neural network;
identify a first accuracy level of the first transcription;
identify a second quantifiable resources value consumed by transcribing the communication using the instances of the transformed neural network;
determine that the first quantifiable resources value is greater than the second quantifiable resources value by over a predetermined resources value threshold;
identify a second accuracy level of the combined transcription;
determine that the first accuracy level is greater than the second accuracy level by over a predetermined accuracy level threshold;
identify a third quantifiable resources value consumed by correcting the combined transcription using the domain-specific corrector;
identify a third accuracy level of the combined transcription upon completion of correcting the combined transcription using the domain-specific corrector;
determine that the third accuracy level is equivalent to or greater than the first accuracy level;
identify a fourth quantifiable resources value, the third quantifiable resources value including the second quantifiable resources value and the third quantifiable resources value; and
determine that the fourth quantifiable resources value is less than the first quantifiable resources value.

### Claim 19 (depends on 17)

The communication transcription system of claim 17 further comprising using the second environment to transcribe incoming communications.

### Claim 20 (depends on 17)

The communication transcription system of claim 17 wherein the audio communication may occur between a human caller and an interactive voice response system.
