# US 2024/0202234 A1 · Keyword variation for querying foreign language audio recordings

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0202234 A1
- Document ID: US20240202234A1
- Status: Published application
- Kind: A1
- Application number: 18/555,077
- Filed: 2022-06-23
- Published: 2024-06-20
- Earliest claimed: 2021-06-23
- Assignees: SRI International
- Applicants: SRI International
- Inventors: Andreas Kathol; Colleen Richey; Victor Abrash; Homin Kwon
- Web page: https://patentgrep.com/patent/US20240202234A1

## CPC

- G06F 16/632: Query formulation
- G10L 15/26: Speech to text systems (G10L15/08 takes precedence)
- G06F 16/65: Clustering; Classification
- G06F 16/683: using metadata automatically derived from the content
- G06F 16/685: {using automatically derived transcript of audio data, e.g. lyrics (speech recognition G10L15/00)}
- G06F 40/58: Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
- G10L 15/005: {Language recognition}
- G10L 15/08: Speech classification or search
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

Techniques are disclosed for searching audio recordings in a second language with a key phrase in a first language. For example, a system as described herein receives a first key phrase in the first language and an audio recording in the second language. The system converts the first key phrase into a second key phrase in the second language. The system processes the second key phrase to produce a second key phrase variant. The system identifies, from a graph of words in the second language generated from the audio recording, instances of the second key phrase or the second key phrase variant within the audio recording. The system displays the identified instances of the second key phrase or the second key phrase variant within the audio recording to enhance searchability of the audio recording in the second language.

## Claims

### Claim at position 1

1: A system comprising:
an input device configured to:
receive a first key phrase in a first language; and
receive an audio recording of speech in a second language different from the first language;
a language conversion unit executed by the processing circuitry, the language conversion unit configured to convert the first key phrase into a second key phrase in the second language;
a word variation unit executed by processing circuitry, the word variation unit configured to process the second key phrase to produce a second key phrase variant in the second language, wherein the second key phrase variant belongs to a same classification as a classification of the second key phrase;
a speech recognition unit configured to process the audio recording to generate a graph of words in the second language;
a query unit configured to query the graph to identify instances of the second key phrase or the second key phrase variant within the audio recording; and
an output device configured to output the identified instances of the second key phrase or the second key phrase variant within the audio recording to enhance searchability of the audio recording of speech in the second language.

### Claim at position 2 (depends on 1)

2: The system of claim 1, wherein to process the second key phrase to produce the second key phrase variant, the word variation unit is configured to apply a stemming operation to the second key phrase to obtain the second key phrase variant.

### Claim at position 3 (depends on 1)

3: The system of claim 1, wherein the classification of the second key phrase comprises a morphology-based grouping.

### Claim at position 4 (depends on 1)

4: The system of claim 1, wherein the classification of the second key phrase comprises a semantic-based grouping.

### Claim at position 5 (depends on 1)

5: The system of claim 1, wherein to convert the first key phrase into the second key phrase, the language conversion unit is configured to perform machine translation of the first key phrase to obtain the second key phrase.

### Claim at position 6 (depends on 1)

6: The system of claim 1, wherein to process the audio recording to generate the graph of words in the second language, the speech recognition unit is configured to:
apply a first speech recognition model to the audio recording to generate an intermediate graph of words in the second language; and
apply a second speech recognition model to the intermediate graph to generate the graph of words in the second language.

### Claim at position 7 (depends on 6)

7: The system of claim 6, wherein a complexity of the first speech recognition model is lower than a complexity of the second speech recognition model.

### Claim at position 8 (depends on 1)

8: The system of claim 1, wherein the output device is further configured to output an indication of a time at which each of the identified instances of the second key phrase or the second key phrase variant occur within the audio recording.

### Claim at position 9 (depends on 1)

9: The system of claim 1, wherein the output device is further configured to output an indication of a count of the identified instances of the second key phrase or the second key phrase variant within the audio recording.

### Claim at position 10 (depends on 1)

10: The system of claim 1, wherein the first key phrase comprises a text string.

### Claim at position 11 (depends on 1)

11: The system of claim 1,
wherein the first key phrase comprises a first plurality of words in the first language,
wherein the second key phrase comprises a second plurality of words in the second language, and
wherein the second key phrase variant comprises a third plurality of words in the second language.

### Claim at position 12 (depends on 1)

12: The system of claim 1,
wherein the first key phrase comprises a first plurality of words in the first language,
wherein the second key phrase comprises a second plurality of words in the second language, and
wherein the second key phrase variant comprises a plurality of second key phrase variants, wherein each second key phrase variant of the plurality of second key phrase variants comprises a third plurality of words in the second language, the third plurality of words of each second key phrase variant different from the third plurality of words of each other second key phrase variant and the second plurality of words of the second key phrase.

### Claim at position 13

13: A method comprising:
receiving, by processing circuitry of a computing system, a first key phrase in a first language;
converting, by the processing circuitry, the first key phrase into a second key phrase in a second language different from the first language;
processing, by the processing circuitry, the second key phrase to produce a second key phrase variant in the second language, wherein the second key phrase variant belongs to a same classification as a classification of the second key phrase;
receiving, by the processing circuitry, an audio recording of speech in the second language;
processing, by the processing circuitry, the audio recording to generate a graph of words in the second language;
querying, by the processing circuitry, the graph to identify instances of the second key phrase or the second key phrase variant within the audio recording; and
outputting, by the processing circuitry, the identified instances of the second key phrase or the second key phrase variant within the audio recording to enhance searchability of the audio recording of speech in the second language.

### Claim at position 14 (depends on 13)

14: The method of claim 13, wherein processing the second key phrase to produce the second key phrase variant comprises applying a stemming operation to the second key phrase to obtain the second key phrase variant.

### Claim at position 15 (depends on 13)

15: The method of claim 13, wherein the classification of the second key phrase comprises at least one of a morphology-based grouping or a semantic-based grouping.

### Claim 16

(canceled)

### Claim at position 17 (depends on 13)

17: The method of claim 13, wherein converting the first key phrase into the second key phrase comprises performing machine translation of the first key phrase to obtain the second key phrase.

### Claim at position 18 (depends on 13)

18: The method of claim 13, wherein processing the audio recording to generate the graph of words in the second language comprises:
applying, by the processing circuitry, a first speech recognition model to the audio recording to generate an intermediate graph of words in the second language; and
applying, by the processing circuitry, a second speech recognition model to the intermediate graph to generate the graph of words in the second language.

### Claim 19

(canceled)

### Claim at position 20 (depends on 13)

20: The method of claim 13, further comprising outputting, by the processing circuitry, at least one of:
an indication of a time at which each of the identified instances of the second key phrase or the second key phrase variant occur within the audio recording; or
an indication of a count of the identified instances of the second key phrase or the second key phrase variant within the audio recording.

### Claim 21

(canceled)

### Claim at position 22 (depends on 13)

22: The method of claim 13,
wherein the first key phrase comprises a first plurality of words in the first language,
wherein the second key phrase comprises a second plurality of words in the second language, and
wherein the second key phrase variant comprises a plurality of second key phrase variants, wherein each second key phrase variant of the plurality of second key phrase variants comprises a third plurality of words in the second language, the third plurality of words of each second key phrase variant different from the third plurality of words of each other second key phrase variant and the second plurality of words of the second key phrase.

### Claim at position 23

23: A non-transitory, computer-readable medium comprising instructions that, when executed, are configured to cause processing circuitry of a computing system to:
receive a first key phrase in a first language;
convert the first key phrase into a second key phrase in a second language different from the first language;
process the second key phrase to produce a second key phrase variant in the second language, wherein the second key phrase variant belongs to a same classification as a classification of the second key phrase;
receive an audio recording of speech in the second language;
process the audio recording to generate a graph of words in the second language;
query the graph to identify instances of the second key phrase or the second key phrase variant within the audio recording; and
output the identified instances of the second key phrase or the second key phrase variant within the audio recording to enhance searchability of the audio recording of speech in the second language.

### Claim at position 24

23-29. (canceled)
