# US 2020/0286485 A1 · Methods and systems for transcription > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2020/0286485 A1 - Document ID: US20200286485A1 - Status: Published application - Kind: A1 - Application number: 16/581,566 - Filed: 2019-09-24 - Published: 2020-09-10 - Earliest claimed: 2018-09-24 - Applicants: Veritone, Inc. - Inventors: Chad Steelberg; Wolf Kohn; Yanfang Shen; Cornelius Raths; Michael Lazarus; Peter Nguyen; Karl Schwamb - Web page: https://patentgrep.com/patent/US20200286485A1 ## CPC - G10L 15/01: Assessment or evaluation of speech recognition systems - G10L 15/26: Speech to text systems (G10L15/08 takes precedence) - G10L 15/02: Feature extraction for speech recognition; Selection of recognition unit - G10L 15/063: {Training} - G10L 15/32: Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems - G10L 2015/025: {Phonemes, fenemes or fenones being the recognition units} - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING ## Abstract Methods and systems for transcribing a media file using a human intelligence task service and/or reinforcement learning are provided. The disclosed systems and methods provide opportunities for a segment of the input media file to be automatically re-analyzed, re-transcribed, and/or modified for re-transcription using a human intelligence task (HIT) service for verification and/or modification of the transcription results. The segment can also be reanalyzed, reconstructed, and re-transcribed using a reinforcement learning enabled transcription model. ## Claims ### Claim 1 A method for transcription, the method comprising: receiving, from a first transcription engine, one or more transcribed portions of a media file; determining a confidence of accuracy value for each of the one or more transcribed portions; identifying, by a transcription analyzer, a first transcribed portion, from the one or more transcribed portions, with a first confidence value below a first predetermined threshold; requesting analysis of the first transcribed portion; receiving, in response to requesting for analysis, an analysis result having a revised-transcription portion of the first transcribed portion, wherein the revised-transcription portion comprises one or more parts of the first transcribed potion that have been revised; and replacing the first transcribed portion with the revised-transcription portion. ### Claim 2 (depends on 1) The method of claim 1 further comprises, after identifying the first transcribed portion and before requesting analysis on the first transcribed portion: sending an audio segment corresponding to the first transcribed portion to a successive plurality of transcription engines; receiving successive transcribed portions from the successive plurality of transcription engines; and replacing the first transcribed portion with one of the received successive transcribed portions based on the second confidence value of the one of the received successive transcribed portions, wherein the revised-transcription portion comprises one or more parts having errors that have been corrected as part of the analysis. ### Claim 3 (depends on 1) The method of claim 1, further comprising: training a machine learning model using a training data set that includes an audio segment corresponding to the first transcribed portion; identifying, by a transcription analyzer, a second transcribed portion having a third confidence value below a second predetermined threshold from the one or more transcribed portions; and using the trained machine learning model, re-transcribing a second audio segment of the media file that corresponds with the second transcribed portion. ### Claim 4 (depends on 1) The method of claim 1, wherein the analysis further comprises: identifying one or more transcription errors in one or more parts of the first transcribed portion; correcting the identified one or more transcription errors in the one or more parts; and labelling the one or more corrected transcription errors in the one or more parts. ### Claim 5 (depends on 1) The method of claim 1, wherein requesting analysis on the first transcribed portion further comprises: constructing a phoneme sequence of an audio segment corresponding to the first transcribed portion based on at least on a reward function; creating a new audio waveform based at least on the constructed phoneme sequence; and generating a new transcription using a transcription engine based on the new audio waveform. ### Claim 6 (depends on 6) The method of claim 6, further comprising: generating a string of cumulants comprising of one or more transcription portions preceding and following the low confidence of accuracy portion, wherein the constructed phoneme sequence is based at least one the string of cumulants; and generating a reward function based at least on one or more characteristics of the transcription engine. ### Claim 7 (depends on 6) The method of claim 6, wherein generating the reward function comprises learning characteristics of the transcription engine by computing a Shannon entropy. ### Claim 8 (depends on 6) The method of claim 6, wherein generating the reward function comprises solving a Bellman equation using backward induction. ### Claim 9 (depends on 9) The method of claim 9, wherein the Bellman equation comprises a Dempster Shafer possibility transition matrix. ### Claim 10 A system for transcription, the system comprising: a memory; one or more processors coupled to the memory, the one or more processors configured to: receive, from a first transcription engine, one or more transcribed portions of a media file; identify, by a transcription analyzer of the conductor, a first transcribed portion, from the one or more transcribed portions, with a confidence value below a predetermined threshold; request analysis of a first audio segment corresponding to the first transcribed portion; receive, in response to request for analysis, an analysis result having a revised-transcription portion of the first audio segment, wherein the revised-transcription portion comprises one or more segments of the first transcribed potion that have been revised; and replace the first transcribed portion with the revised-transcription portion. ### Claim 11 (depends on 11) The system of claim 11, wherein the one or more processors, after identifying the first transcribed portion and before requesting analysis on the first transcribed portion, are further configured to: send the first audio segment to a plurality of transcription engines; receive successive transcribed portions from the plurality of transcription engines; and replace the first transcribed portion with one of the received successive transcribed portions based on the second confidence value of the one of the received successive transcribed portions. ### Claim 12 (depends on 18) The system of claim 18, wherein the one or more processors are further configured to: train a machine learning model using a training data set from the low-confidence database; identify, by a transcription analyzer, a second transcribed portion having a third confidence value below a second predetermined threshold from the one or more transcribed portions; and using the trained machine learning model, re-transcribe a second audio segment of the media file that corresponds with the second transcribed portion. ### Claim 13 (depends on 11) The system of claim 11, further comprising: a ground truth engine configured to: identify one or more transcription errors in one or more parts of the first transcribed portion; correct the identified one or more transcription errors in the one or more parts; and label the one or more corrected transcription errors in the one or more parts. ### Claim 14 (depends on 10) The system of claim 10, wherein request analysis on the first transcribed portion further comprises instructions that cause the one or more processor to: construct a phoneme sequence of an audio segment corresponding to the first transcribed portion based on at least on a reward function; create a new audio waveform based at least on the constructed phoneme sequence; and generate a new transcription using a transcription engine based on the new audio waveform. ### Claim 15 (depends on 10) The system of claim 10, wherein the one or more processors are further configured to: generate a string of cumulants comprising of one or more transcription portions preceding and following the low confidence of accuracy portion, wherein the constructed phoneme sequence is based at least one the string of cumulants; and generate a reward function based at least on one or more characteristics of the transcription engine. ### Claim 16 (depends on 15) The system of claim 15, wherein generate the reward function comprises learning characteristics of the transcription engine by computing a Shannon entropy. ### Claim 17 (depends on 15) The system of claim 15, wherein generate the reward function comprises solving a Bellman equation using backward induction. ### Claim 18 (depends on 17) The system of claim 17, wherein the Bellman equation comprises a Dempster Shafer possibility transition matrix. ### Claim 19 A method for transcription, the method comprising: receiving one or more transcribed portions of a media file; determining a confidence of accuracy value for each of the one or more transcribed portions; identifying a first transcribed portion that has a first confidence value below a predetermined threshold; constructing a phoneme sequence of an audio segment corresponding to the first transcribed portion based on at least on a reward function; creating a new audio waveform based at least on the constructed phoneme sequence; generating a new transcription using a transcription engine based on the new audio waveform; and replacing the first transcribed portion with the new transcription. ### Claim 20 (depends on 19) The method of claim 19 wherein generating the new transcription further comprises: sending the new audio waveform to a plurality of transcription engines; receiving transcription results from the plurality of transcription engines; and replacing the first transcribed portion with one of the transcription results based on a second confidence value, wherein each transcription result includes a confidence value.