# US 2020/0394258 A1 · Generation of edited transcription for speech audio

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0394258 A1
- Document ID: US20200394258A1
- Status: Published application
- Kind: A1
- Application number: 16/442,454
- Filed: 2019-06-15
- Published: 2020-12-17
- Earliest claimed: 2019-06-15
- Assignees: SoundHound, Inc.
- Applicants: SoundHound, Inc.
- Inventors: Haoliang Chen; Junru Ren
- Web page: https://patentgrep.com/patent/US20200394258A1

## CPC

- G06F 40/166: Editing, e.g. inserting or deleting
- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G06F 40/284: Lexical analysis, e.g. tokenisation or collocates
- G10L 15/1815: {Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning}
- G10L 15/26: Speech to text systems (G10L15/08 takes precedence)
- G10L 2015/228: {of application context}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

The present disclosure is intended to provide method, manufacture products, and apparatuses for generating an edited transcription of a speech audio in a SR-NLU system. A method for generating an edited transcription of a speech audio may include performing automatic speech recognition on the speech audio to produce a transcription having one or more tokens; interpreting the transcription according to each of a plurality of natural language domains to produce a plurality of interpretation results; identifying, based on the plurality of interpretation results, a natural language domain that matches the transcription; and replacing a token of interest in the transcription with a replacement token according to a predefined mapping specific to the identified natural language domain to generate the edited transcription of the speech audio.

## Claims

### Claim 1

A method for generating an edited transcription of a speech audio, the method comprising:
performing automatic speech recognition on the speech audio to produce a transcription having one or more tokens;
interpreting the transcription according to each of a plurality of natural language domains to produce a plurality of interpretation results;
identifying, based on the plurality of interpretation results, a natural language domain that matches the transcription; and
replacing a token of interest in the transcription with a replacement token according to a predefined mapping specific to the identified natural language domain to generate the edited transcription of the speech audio.

### Claim 2 (depends on 1)

The method of claim 1, wherein the speech audio is to be updated in real time, and the method further comprises:
performing the automatic speech recognition on the updated speech audio to produce an updated transcription;
detecting if the updated transcription includes a new token; and
repeating the transcription interpretation, the domain identification and the token replacement for the updated transcription to generate an updated edited transcription once it is detected that the updated transcription includes the new token.

### Claim 3 (depends on 1)

The method of claim 1, wherein the transcription comprises multiple transcription hypotheses, and interpreting the transcription comprises interpreting the multiple transcription hypotheses according to each of the plurality of natural language domains.

### Claim 4 (depends on 1)

The method of claim 1, wherein the predefined mapping comprises a programmatic mapping.

### Claim 5 (depends on 4)

The method of claim 4, wherein the predefined mapping further comprises a simple token replacement mapping, and the simple token replacement mapping and the programmatic mapping are stored separately in a memory, and the token replacement according to the simple token replacement mapping and the token replacement according to the programmatic mapping are performed simultaneously on separate processing threads.

### Claim 6 (depends on 4)

The method of claim 4, wherein the predefined mapping further comprises a simple token replacement mapping, and the simple token replacement mapping is stored as a search tree in the memory.

### Claim 7 (depends on 4)

The method of claim 4, wherein the programmatic mapping comprises a regular expression mapping.

### Claim 8 (depends on 1)

The method of claim 1, wherein the predefined mapping is editable to add, revise or delete a mapping entry in the predefined mapping.

### Claim 9 (depends on 1)

The method of claim 1, wherein the predefined mapping is dynamically retrieved over a network from a natural language understanding (NLU) interpreting server.

### Claim 10 (depends on 1)

The method of claim 1, wherein the one or more tokens comprise alphabetic words including English words, logographic characters including Chinese characters, or discernable elemental units of other types of writing systems.

### Claim 11 (depends on 1)

The method of claim 1, wherein:
the replacement token is an abbreviation of the token of interest;
the token of interest is a textual expression of a number and the replacement token is the number;
the token of interest has a vulgar meaning and the replacement token is a polite synonym of the token of interest;
the token of interest is a loan word and the replacement token is a synonym native to a language of the speech audio; or
the replacement token has a same pronunciation as the token of interest and a more proper written form than the token of interest in the identified natural language domain.

### Claim 12 (depends on 1)

The method of claim 1, wherein the token of interest and the replacement token are encoded in different character encodings.

### Claim 13 (depends on 1)

The method of claim 1, further comprising:
tagging the replacement token within the edited transcription so as to enable rendering the edited transcription with the replacement token being distinguishable.

### Claim 14

A non-transitory computer readable medium storing code that, if executed by one or more processors, causes the one or more processors to:
perform automatic speech recognition on the speech audio to produce a transcription having one or more tokens;
interpret the transcription according to each of a plurality of natural language domains to produce a plurality of interpretation results;
identify, based on the plurality of interpretation results, a natural language domain that matches the transcription; and
replace a token of interest in the transcription with a replacement token according to a predefined mapping specific to the identified natural language domain to generate the edited transcription of the speech audio.

### Claim 15 (depends on 14)

The non-transitory computer readable medium of claim 14, wherein the speech audio is to be updated in real time, and the code, if executed by the one or more processors, causes the one or more processors further to:
perform the automatic speech recognition on the updated speech audio to produce an updated transcription;
detect if the updated transcription includes a new token; and
repeat the transcription interpretation, the domain identification and the token replacement for the updated transcription to generate an updated edited transcription once it is detected that the updated transcription includes the new token.

### Claim 16 (depends on 14)

The non-transitory computer readable medium of claim 14, wherein the transcription comprises multiple transcription hypotheses, and the code, if executed by the one or more processors, causes the one or more processors to interpret the transcription by interpreting the multiple transcription hypotheses according to each of the plurality of natural language domains.

### Claim 17 (depends on 14)

The non-transitory computer readable medium of claim 14, wherein the predefined mapping comprises a programmatic mapping.

### Claim 18 (depends on 17)

The non-transitory computer readable medium of claim 17, wherein the predefined mapping further comprises a simple token replacement mapping, and the simple token replacement mapping and the programmatic mapping are stored separately in a memory, and the token replacement according to the simple token replacement mapping and the token replacement according to the programmatic mapping are performed simultaneously on separate processing threads.

### Claim 19 (depends on 17)

The non-transitory computer readable medium of claim 17, wherein the predefined mapping further comprises a simple token replacement mapping, and the simple token replacement mapping is stored as a search tree in the memory.

### Claim 20 (depends on 17)

The non-transitory computer readable medium of claim 17, wherein the programmatic mapping comprises a regular expression mapping.

### Claim 21 (depends on 14)

The non-transitory computer readable medium of claim 14, wherein the predefined mapping is editable to add, revise or delete a mapping entry in the predefined mapping.

### Claim 22 (depends on 14)

The non-transitory computer readable medium of claim 14, wherein the predefined mapping is dynamically retrieved over a network from a natural language understanding (NLU) interpreting server.

### Claim 23 (depends on 14)

The non-transitory computer readable medium of claim 14, wherein the one or more tokens comprise alphabetic words including English words, logographic characters including Chinese characters, or discernable elemental units of other types of writing systems.

### Claim 24 (depends on 14)

The non-transitory computer readable medium of claim 14, wherein:
the replacement token is an abbreviation of the token of interest;
the token of interest is a textual expression of a number and the replacement token is the number;
the token of interest has a vulgar meaning and the replacement token is a polite synonym of the token of interest;
the token of interest is a loan word and the replacement token is a synonym native to a language of the speech audio; or
the replacement token has a same pronunciation as the token of interest and a more proper written form than the token of interest in the identified natural language domain.

### Claim 25 (depends on 14)

The non-transitory computer readable medium of claim 14, wherein the token of interest and the replacement token are encoded in different character encodings.

### Claim 26 (depends on 14)

The non-transitory computer readable medium of claim 14, wherein the code, if executed by the one or more processors, causes the one or more processors further to:
tag the replacement token within the edited transcription so as to enable rendering the edited transcription with the replacement token being distinguishable.

### Claim 27

An apparatus for generating an edited transcription of a speech audio, the apparatus comprising:
a memory; and
a processor to access the memory via a memory interface,
wherein the processor is configured to:
perform automatic speech recognition on the speech audio to produce a transcription having one or more tokens;
interpret the transcription according to each of a plurality of natural language domains to produce a plurality of interpretation results;
identify, based on the plurality of interpretation results, a natural language domain that matches the transcription; and
replace a token of interest in the transcription with a replacement token according to a predefined mapping specific to the identified natural language domain to generate the edited transcription of the speech audio,
wherein the memory is to store a plurality of predefined mappings each of which is respectively specific to each of the plurality of natural language domains.

### Claim 28 (depends on 27)

The apparatus of claim 27, wherein the speech audio is to be updated in real time, and the processor is further configured to:
perform the automatic speech recognition on the updated speech audio to produce an updated transcription;
detect if the updated transcription includes a new token; and
repeat the transcription interpretation, the domain identification and the token replacement for the updated transcription to generate an updated edited transcription once it is detected that the updated transcription includes the new token.

### Claim 29 (depends on 27)

The apparatus of claim 27, wherein the transcription comprises multiple transcription hypotheses, and the processor is configured to interpret the transcription by interpreting the multiple transcription hypotheses according to each of the plurality of natural language domains.

### Claim 30 (depends on 27)

The apparatus of claim 27, wherein the predefined mapping comprises a programmatic mapping.

### Claim 31 (depends on 30)

The apparatus of claim 30, wherein the predefined mapping further comprises a simple token replacement mapping, and the simple token replacement mapping and the programmatic mapping are stored separately in a memory, and the token replacement according to the simple token replacement mapping and the token replacement according to the programmatic mapping are performed simultaneously on separate processing threads.

### Claim 32 (depends on 30)

The apparatus of claim 30, wherein the predefined mapping further comprises a simple token replacement mapping, and the simple token replacement mapping is stored as a search tree in the memory.

### Claim 33 (depends on 30)

The apparatus of claim 30, wherein the programmatic mapping comprises a regular expression mapping.

### Claim 34 (depends on 27)

The apparatus of claim 27, wherein the predefined mapping is editable to add, revise or delete a mapping entry in the predefined mapping.

### Claim 35 (depends on 27)

The apparatus of claim 27, wherein the predefined mapping is dynamically retrieved over a network from a natural language understanding (NLU) interpreting server.

### Claim 36 (depends on 27)

The apparatus of claim 27, wherein the one or more tokens comprise alphabetic words including English words, logographic characters including Chinese characters, or discernable elemental units of other types of writing systems.

### Claim 37 (depends on 27)

The apparatus of claim 27, wherein:
the replacement token is an abbreviation of the token of interest;
the token of interest is a textual expression of a number and the replacement token is the number;
the token of interest has a vulgar meaning and the replacement token is a polite synonym of the token of interest;
the token of interest is a loan word and the replacement token is a synonym native to a language of the speech audio; or
the replacement token has a same pronunciation as the token of interest and a more proper written form than the token of interest in the identified natural language domain.

### Claim 38 (depends on 27)

The apparatus of claim 27, wherein the token of interest and the replacement token are encoded in different character encodings.

### Claim 39 (depends on 27)

The apparatus of claim 27, wherein the processor is further configured to:
tag the replacement token within the edited transcription so as to enable rendering the edited transcription with the replacement token being distinguishable.
