# US 2024/0256599 A1 · Responding to queries with voice recordings

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0256599 A1
- Document ID: US20240256599A1
- Status: Published application
- Kind: A1
- Application number: 18/631,952
- Filed: 2024-04-10
- Published: 2024-08-01
- Earliest claimed: 2022-05-23
- Assignees: Google Inc.
- Applicants: Google LLC
- Inventors: Sowmya Subramanian; Benton Davis DeLoache; Lauren Clark; Rami Banna; Igor Benko
- Web page: https://patentgrep.com/patent/US20240256599A1

## CPC

- G06F 16/635: Filtering based on additional data, e.g. user or group profiles
- G06F 40/35: Discourse or dialogue representation
- G06F 16/686: {using information manually generated, e.g. tags, keywords, comments, title or artist information, time, location or usage information, user ratings}
- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L 2015/223: {Execution procedure of a spoken command}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

Implementations are provided for providing responsive audio recordings to user queries that are prerecorded by human beings, rather than generated automatically using speech synthesis processing. In various implementations, a query provided by a user at an input component of a computing device may be used to search a corpus of voice recordings. From the searching, a plurality of candidate responsive voice recordings may be identified and ranked based on measures of credibility associated with speakers that created the candidate responsive voice recordings. Based on the ranking, one or more of the plurality of candidate responsive voice recordings may be provided for presentation to the user at an output component of the same computing device or a different computing device.

## Claims

### Claim 1

A method implemented using one or more processors, comprising:
receiving a query provided by a user at an input component of a computing device;
searching a corpus of voice recordings based on the query;
identifying, from the searching, a plurality of candidate responsive voice recordings;
determining measures of credibility associated with respective speakers of the plurality of candidate responsive voice recordings, wherein the respective measure of credibility for each speaker is determined by grouping semantic embeddings generated from voice recordings created by that speaker into one or more clusters, and analyzing the one or more clusters for semantic consistency;
ranking the plurality of candidate responsive voice recordings based on the measures of credibility; and
based on the ranking, selecting, for presentation to the user at an output component of the same computing device or a different computing device, one or more of the plurality of candidate responsive voice recordings.

### Claim 2 (depends on 1)

The method of claim 1, wherein the semantic embeddings generated from the voice recordings are grouped into clusters based on cosine similarity or Euclidean distance between the semantic embeddings.

### Claim 3 (depends on 1)

The method of claim 1, wherein the one or more clusters comprise a plurality of clusters, and analyzing the one or more clusters for semantic consistency comprises determining whether the plurality of clusters are semantically consistent.

### Claim 4 (depends on 1)

The method of claim 1, wherein analyzing the one or more clusters for semantic consistency comprises analyzing a size of the one or more clusters.

### Claim 5 (depends on 1)

The method of claim 1, wherein the searching includes searching transcriptions of the voice recordings stored in association with the voice recordings.

### Claim 6 (depends on 1)

The method of claim 1, wherein the searching further comprises searching questions stored in association with, and answered by, voice recordings of the corpus.

### Claim 7 (depends on 6)

The method of claim 6, wherein searching questions comprises determining distances between an embedding generated from the query from the user and one or more embeddings generated from one or more questions that are answered by one or more voice recordings of the corpus.

### Claim 8

A method implemented using one or more processors, comprising:
receiving a query provided by a user at an input component of a computing device;
searching a corpus of voice recordings based on the query;
identifying, from the searching, a plurality of candidate responsive voice recordings;
determining measures of credibility associated with the plurality of candidate responsive voice recordings, wherein the respective measure of credibility for each candidate responsive voice recording is determined based on:
its relationship with a cluster of semantic embeddings generated from semantically similar voice recordings that are responsive to a crowd-sourced question, and
a measure of credibility attributed to the cluster;
ranking the plurality of candidate responsive voice recordings based on the measures of credibility; and
based on the ranking, selecting, for presentation to the user at an output component of the same computing device or a different computing device, one or more of the plurality of candidate responsive voice recordings.

### Claim 9 (depends on 8)

The method of claim 8, wherein the relationship is determined using one or more of Euclidian distance, cosine similarity, or dot product.

### Claim 10 (depends on 8)

The method of claim 8, wherein the measure of credibility attributed to the cluster is based on a size of the cluster relative to other clusters of semantic embeddings generated from semantically similar voice recordings that are also responsive to the crowd-sourced question.

### Claim 11 (depends on 8)

The method of claim 8, wherein the measure of credibility attributed to the cluster is based on interactions by others with voice recordings used to generate the semantic embeddings of the cluster.

### Claim 12 (depends on 8)

The method of claim 8, wherein the semantic embeddings are grouped into the cluster based on cosine similarity or Euclidean distance between the semantic embeddings.

### Claim 13 (depends on 8)

The method of claim 8, wherein the searching includes searching transcriptions of the voice recordings stored in association with the voice recordings.

### Claim 14 (depends on 8)

The method of claim 8, wherein the searching further comprises searching questions stored in association with, and answered by, voice recordings of the corpus.

### Claim 15

A system comprising one or more processors and memory storing instructions that, in response to execution by the one or more processors, cause the one or more processors to:
receive a query provided by a user at an input component of a computing device;
search a corpus of voice recordings based on the query;
identify, from the searching, a plurality of candidate responsive voice recordings;
determine measures of credibility associated with respective speakers of the plurality of candidate responsive voice recordings, wherein the respective measure of credibility for each speaker is determined by grouping semantic embeddings generated from voice recordings created by that speaker into one or more clusters, and analyzing the one or more clusters for semantic consistency;
rank the plurality of candidate responsive voice recordings based on the measures of credibility; and
based on the ranking, select, for presentation to the user at an output component of the same computing device or a different computing device, one or more of the plurality of candidate responsive voice recordings.

### Claim 16 (depends on 15)

The system of claim 15, wherein the semantic embeddings generated from the voice recordings are grouped into clusters based on cosine similarity or Euclidean distance between the semantic embeddings.

### Claim 17 (depends on 15)

The system of claim 15, wherein the one or more clusters comprise a plurality of clusters, and the instructions to analyze the one or more clusters for semantic consistency include instructions to determine whether the plurality of clusters are semantically consistent.

### Claim 18 (depends on 15)

The system of claim 15, wherein the instructions to analyze the one or more clusters for semantic consistency include instructions to analyze a size of the one or more clusters.

### Claim 19 (depends on 15)

The system of claim 15, wherein the instructions to search include instructions to search transcriptions of the voice recordings stored in association with the voice recordings.

### Claim 20 (depends on 15)

The system of claim 15, wherein the instructions to search further include instructions to search questions stored in association with, and answered by, voice recordings of the corpus.
