# US 12,192,409 B2 · Categorizing audio calls based on machine learning models > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,192,409 B2 - Document ID: US12192409B2 - Status: Granted - Kind: B2 - Application number: 16/903,082 - Filed: 2020-06-16 - Published: 2021-08-26 - Granted: 2025-01-07 - Earliest claimed: 2020-02-21 - Assignees: Sap SE - Applicants: Sap SE - Inventors: Anh Burton; Hans-Martin Will; James Tian; Andreas Kurt Pursche; Jeff Dixon; Matthew Berg; Thomas Obermeyer - Web page: https://patentgrep.com/patent/US12192409B2 ## CPC - H04M 3/5235: {Dependent on call type or called number [DNIS]} - G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback} - G06F 40/30: Semantic analysis - G06N 20/00: Machine learning - G06Q 30/016: After-sales - G10L 15/26: Speech to text systems (G10L15/08 takes precedence) - H04M 3/42221: {Conversation recording systems (at the subscriber's set H04M1/656)} - H04M 2201/40: using speech recognition - H04M 2203/301: Management of recordings - H04M 2203/558: Databases - H04M: TELEPHONIC COMMUNICATION (circuits for controlling other apparatus via a telephone cable and not involving telephone switching apparatus G08) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING ## Abstract Some embodiments provide a non-transitory machine-readable medium that stores a program. The program receives a set of audio files. Each audio file in the set of audio files includes audio from an audio call. The program further truncates each audio file in the set of audio files to a defined call length. For each audio call in the set of audio calls, the program also receives a transcript of the audio call based on the audio file of the audio call. For each audio call in the set of audio calls, the program further uses the transcript of the audio call as input to a machine learning model for the machine learning model to predict a category from a plurality of categories that is associated with the audio call. ## Claims ### Claim 1 A non-transitory machine-readable medium 1 storing a program executable by at least one processing unit of a device, the program comprising sets of instructions for: receiving a set of audio files, each audio file in the set of audio files comprising audio from an audio call; truncating each audio file in the set of audio files to a defined call length; for each audio call in the set of audio calls, receiving a transcript of the audio call based on the truncated audio file of the audio call; and for each audio call in the set of audio calls, using the transcript of the audio call as input to a machine learning model for the machine learning model to predict a category from a plurality of categories that is associated with the audio call. ### Claim 2 (depends on 1) The non-transitory machine-readable medium of claim 1, wherein the program further comprises sets of instructions for: for each audio file in the set of audio files, sending the audio file to a third-party transcription service; and for each audio file in the set of audio files, receiving the transcript of the audio file from the third-party transcription service, wherein the third-party transcription service generates the transcript by converting the audio file to text using a speech-to-text algorithm. ### Claim 3 (depends on 1) The non-transitory machine-readable medium of claim 1, wherein the program further comprises sets of instructions for: receiving a list of the set of audio calls from a third-party system configured to manage audio calls received at customer support centers; sending the third-party system a request for metadata associated with the set of audio calls, the metadata comprising locations of the set of audio files of the audio calls; and retrieving the set of audio files of the set of audio calls from the locations, wherein receiving the set of audio files comprises retrieving the set of audio files. ### Claim 4 (depends on 3) The non-transitory machine-readable medium of claim 3, wherein the program further comprises a set of instructions for storing the set of metadata associated with the set of audio calls and the categories associated with the set of audio calls in a storage. ### Claim 5 (depends on 4) The non-transitory machine-readable medium of claim 4, wherein the set of transcripts of the set of audio calls and the categories associated with the set of transcripts of the set of audio calls stored in the storage is used by a cloud computing system to generate visualizations for a client device. ### Claim 6 (depends on 1) The non-transitory machine-readable medium of claim 1, wherein each audio file in the set of audio files comprise a recording of a customer support session, wherein each category in the plurality of categories represents a closure classification for a particular customer support session. ### Claim 7 (depends on 1) The non-transitory machine-readable medium of claim 1, wherein the machine learning model is a transformer-based model. ### Claim 8 A method comprising: receiving a set of audio files, each audio file in the set of audio files comprising audio from an audio call; truncating each audio file in the set of audio files to a defined call length; for each audio call in the set of audio calls, receiving a transcript of the audio call based on the truncated audio file of the audio call; and for each audio call in the set of audio calls, using the transcript of the audio call as input to a machine learning model for the machine learning model to predict a category from a plurality of categories that is associated with the audio call. ### Claim 9 (depends on 8) The method of claim 8 further comprising: for each audio file in the set of audio files, sending the audio file to a third-party transcription service; and for each audio file in the set of audio files, receiving from the transcript of the audio file from the third-party transcription service, wherein the third-party transcription service generates the transcript by converting the audio file to text using a speech-to-text algorithm. ### Claim 10 (depends on 8) The method of claim 8 further comprising: receiving a list of the set of audio calls from a third-party system configured to manage audio calls received at customer support centers; sending the third-party system a request for metadata associated 4 with the set of audio calls, the metadata comprising locations of the set of audio files of the audio calls; and retrieving the set of audio files of the set of audio calls from the locations, wherein receiving the set of audio files comprises retrieving the set of audio files. ### Claim 11 (depends on 10) The method of claim 10 further comprising storing the set of metadata associated with the set of audio calls and the categories associated with the set of audio calls in a storage. ### Claim 12 (depends on 11) The method of claim 11, wherein the set of transcripts of the set of audio calls and the categories associated with the set of transcripts of the set of audio calls stored in the storage is used by a cloud computing system to generate visualizations for a client device. ### Claim 13 (depends on 8) The method of claim 8, wherein each audio file in the set of audio files comprise a recording of a customer support session, wherein each category in the plurality of categories represents a closure classification for a particular customer support session. ### Claim 14 (depends on 8) The method of claim 8, wherein the machine learning model is a transformer-based model. ### Claim 15 A system comprising: a set of processing units; and a non-transitory machine-readable medium storing instructions that when executed by at least one processing unit in the set of processing units cause the at least one processing unit to: receive a set of audio files, each audio file in the set of audio files comprising audio from an audio call; truncate each audio file in the set of audio files to a defined call length; for each audio call in the set of audio calls, receive a transcript of the audio call based on the truncated audio file of the audio call; and for each audio call in the set of audio calls, use the transcript of the audio call as input to a machine learning model for the machine learning model to predict a category from a plurality of categories that is associated with the audio call. ### Claim 16 (depends on 15) The system of claim 15, wherein the instructions further 1 cause the at least one processing unit to: for each audio file in the set of audio files, send the audio file to a third-party transcription service; and for each audio file in the set of audio files, receive from the transcript of the audio file from the third-party transcription service, wherein the third-party transcription service generates the transcript by converting the audio file to text using a speech-to-text algorithm. ### Claim 17 (depends on 15) The system of claim 15, wherein the instructions further cause the at least one processing unit to: receive a list of the set of audio calls from a third-party system configured to manage audio calls received at customer support centers; send the third-party system a request for metadata associated with the set of audio calls, the metadata comprising locations of the set of audio files of the audio calls; and retrieve the set of audio files of the set of audio calls from the locations, wherein receiving the set of audio files comprises retrieving the set of audio files. ### Claim 18 (depends on 17) The system of claim 17, wherein the instructions further cause the at least one processing unit to store the set of metadata associated with the set of audio calls and the categories associated with the set of audio calls in a storage. ### Claim 19 (depends on 18) The system of claim 18, wherein the set of transcripts of the set of audio calls and the categories associated with the set of transcripts of the set of audio calls stored in the storage is used by a cloud computing system to generate visualizations for a client device. ### Claim 20 (depends on 15) The system of claim 15, wherein each audio file in the set of audio files comprise a recording of a customer support session, wherein each category in the plurality of categories represents a closure classification for a particular customer support session.