# US 12,190,867 B2 · Keyword detection for audio content

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,190,867 B2
- Document ID: US12190867B2
- Status: Granted
- Kind: B2
- Application number: 17/804,603
- Filed: 2022-05-31
- Published: 2023-10-26
- Granted: 2025-01-07
- Earliest claimed: 2022-04-20
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Zvi Figov
- Web page: https://patentgrep.com/patent/US12190867B2

## CPC

- G10L 15/08: Speech classification or search
- G06F 40/279: Recognition of textual entities
- G06F 40/30: Semantic analysis
- G06F 40/40: Processing or translation of natural language (natural language analysis G06F40/20; semantic analysis G06F40/30)
- G10L 15/04: Segmentation; Word boundary detection
- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L 25/57: for processing of video signals
- G10L 2015/088: {Word spotting}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Examples of the present disclosure describe improved systems and methods for detecting keywords in audio content. In one example implementation, audio content is segmented into one or more audio segments. One or more text segments is generated, each text segment corresponding to each of the audio segments. For each text segment, one or more phrase candidate values is generated using a textual analysis, and one or more sentence embedding values is generated using a sentence embedding analysis. Next, an average sentence embedding value is calculated using the one or more sentence embedding values. Each of the one or more phrase candidate values is compared to the average sentence embedding value. Each phrase candidate value having a comparison value above a threshold value is labeled as representing a keyword.

## Claims

### Claim 1

A method for detecting keywords for audio content, the method comprising:
segmenting the audio content into a plurality of audio segments;
generating a plurality of text segments corresponding to the plurality of audio segments;
generating a plurality of phrase candidate values using a textual analysis of the plurality of text segments;
generating a plurality of sentence embedding values using a sentence embedding analysis of the plurality of text segments;
calculating an average sentence embedding value using the plurality of sentence embedding values;
comparing each phrase candidate value of the plurality of phrase candidate values to the average sentence embedding value;
labeling each phrase candidate value having a comparison value above a threshold value as a keyword; and
presenting the keyword as a stream of the audio content progresses.

### Claim 2 (depends on 1)

The method of claim 1, wherein the detecting keywords for audio content occurs during playback of the audio content.

### Claim 3 (depends on 1)

The method of claim 1, wherein the average sentence embedding value approaches the threshold value as playback of the audio content progresses.

### Claim 4 (depends on 1)

The method of claim 1, wherein each text segment of the plurality of text segments corresponds to an audio segment in the plurality of text segments.

### Claim 5 (depends on 1)

The method of claim 1, wherein video content comprises the audio content.

### Claim 6 (depends on 1)

The method of claim 1, wherein the textual analysis comprises a stopword analysis.

### Claim 7 (depends on 1)

The method of claim 1, wherein the plurality of sentence embedding values comprise a plurality of vectors, respectively.

### Claim 8 (depends on 1)

The method of claim 1, wherein the threshold value is a predetermined value based on relevance.

### Claim 9

A system for detecting keywords for audio content, the system comprising:
a language conversion module stored in memory and executable to segment audio content into a plurality of audio segments and to generate a plurality of text segments, each text segment of the plurality of text segments corresponding to a particular audio segment of the plurality of audio segments;
a phrase generator module stored in the memory and executable to: generate a plurality of phrase candidate values using a textual analysis of the plurality of text segments, generate a plurality of sentence embedding values using a sentence embedding analysis of the plurality of text segments, and calculate an average sentence embedding value using the plurality of sentence embedding values; and
a candidate extraction module stored in the memory and executable to: compare each of the plurality of phrase candidate values to the average sentence embedding value, and to label each phrase candidate value having a comparison value above a threshold value as representing a keyword, and present the keyword as a stream of the audio content progresses.

### Claim 10 (depends on 9)

The system of claim 9, wherein the average sentence embedding value approaches the threshold value as playback of the audio content progresses.

### Claim 11 (depends on 9)

The system of claim 9, wherein the language conversion module, the phrase generator module, and the phrase candidate extraction module are stored in the memory on a personal computing device.

### Claim 12 (depends on 9)

The system of claim 9, wherein the audio content comprises a livestream.

### Claim 13 (depends on 9)

The system of claim 9, wherein each text segment of the plurality of text segments exists in a one-to-one relationship with an audio segment in the plurality of audio segments.

### Claim 14

A device comprising:
a processor; and
memory coupled to the processor, the memory comprising computer executable instructions that, when executed by the processor, perform operations comprising:
segmenting audio content into one or more audio segments;
generating one or more text segments, each text segment corresponding to each of the audio segments;
for each text segment:
generating one or more phrase candidate values using a textual analysis; and
generating one or more sentence embedding values using a sentence embedding analysis;
calculating an average sentence embedding value using the one or more sentence embedding values;
comparing each of the one or more phrase candidate values to the average sentence embedding value;
labeling each phrase candidate value having a comparison value above a threshold value as representing a keyword; and
presenting the keyword as a stream of the audio content progresses.

### Claim 15 (depends on 14)

The device of claim 14, wherein the audio content comprises a livestream.

### Claim 16 (depends on 14)

The device of claim 14, wherein the operations are performed during playback of the audio content, and wherein the average sentence embedding value approaches the threshold value as playback of the audio content progresses.

### Claim 17 (depends on 14)

The device of claim 14, wherein the textual analysis comprises a stopword analysis.

### Claim 18 (depends on 14)

The device of claim 14, wherein the one or more sentence embedding values comprise one or more vectors.

### Claim 19 (depends on 14)

The device of claim 14, wherein the threshold value is a predetermined value based on relevance.

### Claim 20 (depends on 14)

The device of claim 14, wherein the processor and the memory are comprised on a personal computing device.