# US 2026/0300386 A1 · Audio question answering with grounding

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0300386 A1
- Document ID: US20260300386A1
- Status: Published application
- Kind: A1
- Application number: 19/091,750
- Filed: 2025-03-26
- Published: 2026-10-01
- Earliest claimed: 2025-03-26
- Assignees: QUALCOMM Incorporated
- Applicants: QUALCOMM Incorporated
- Inventors: Yinyi GUO; Arvind Krishna SRIDHAR; Rehana MAHFUZ; Erik VISSER
- Web page: https://patentgrep.com/patent/US20260300386A1

## CPC

- G06F 16/634: {Query by example, e.g. query by humming}
- G06F 16/683: using metadata automatically derived from the content
- G06F 16/686: {using information manually generated, e.g. tags, keywords, comments, title or artist information, time, location or usage information, user ratings}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Systems and techniques are described herein for processing audio data. For instance, a method for processing audio data is provided. The method may include encoding audio data to generate first audio embeddings; adapting the first audio embeddings to generate second audio embeddings; combining the first audio embeddings and the second audio embeddings to generate third audio embeddings; projecting the third audio embeddings to generate text projections; generating audio metadata based on the second audio embeddings; and processing the text projections, the audio metadata, and a query using a machine-learning model to generate a response to the query.

## Claims

### Claim 1

An apparatus for processing audio data, the apparatus comprising:
one or more memories configured to store the audio data; and
one or more processors coupled to the one or more memories and configured to:
encode the audio data to generate first audio embeddings;
adapt the first audio embeddings to generate second audio embeddings;
combine the first audio embeddings and the second audio embeddings to generate third audio embeddings;
project the third audio embeddings to generate text projections;
generate audio metadata based on the second audio embeddings; and
process the text projections, the audio metadata, and a query using a machine-learning model to generate a response to the query.

### Claim 2 (depends on 1)

The apparatus of claim 1, wherein the one or more processors are configured to generate caption data based on the first audio embeddings, wherein the response is generated by further processing the caption data using the machine-learning model.

### Claim 3 (depends on 1)

The apparatus of claim 1, wherein the one or more processors are configured to adapt the first audio embeddings using an audio grounding adapter trained through a contrastive-learning process.

### Claim 4 (depends on 1)

The apparatus of claim 1, wherein the one or more processors are configured to adapt the first audio embeddings using an audio grounding adapter trained based on a knowledge-distillation process using a contrastive-learning audio pretrained audio encoder.

### Claim 5 (depends on 1)

The apparatus of claim 1, wherein the one or more processors are configured to combine the first audio embeddings and the second audio embeddings using a cross-attention transformer.

### Claim 6 (depends on 1)

The apparatus of claim 1, wherein the one or more processors are configured to project the third audio embeddings using a projector trained to generate text projections based on audio embeddings.

### Claim 7 (depends on 1)

The apparatus of claim 1, wherein the audio metadata comprises a list of audio events and corresponding timestamps.

### Claim 8 (depends on 1)

The apparatus of claim 1, wherein the one or more processors are configured to generate the audio metadata based on at least one prior user input.

### Claim 9 (depends on 1)

The apparatus of claim 1, wherein the one or more processors are configured to process a prior user input using a text encoder to generate input embeddings, wherein the audio metadata is generated further based on the input embeddings.

### Claim 10 (depends on 1)

The apparatus of claim 1, further comprising a microphone to record audio, wherein the one or more processors is configured to convert the recorded audio into the query.

### Claim 11 (depends on 1)

The apparatus of claim 1, wherein the one or more processors are configured to convert the response into an audio signal, the apparatus further comprising a speaker to play the audio signal.

### Claim 12

A method for processing audio data, the method comprising:
encoding audio data to generate first audio embeddings;
adapting the first audio embeddings to generate second audio embeddings;
combining the first audio embeddings and the second audio embeddings to generate third audio embeddings;
projecting the third audio embeddings to generate text projections;
generating audio metadata based on the second audio embeddings; and
processing the text projections, the audio metadata, and a query using a machine-learning model to generate a response to the query.

### Claim 13 (depends on 12)

The method of claim 12, further comprising generating caption data based on the first audio embeddings, wherein the response is generated by further processing the caption data using the machine-learning model.

### Claim 14 (depends on 12)

The method of claim 12, wherein the first audio embeddings are adapted using an audio grounding adapter trained through a contrastive-learning process.

### Claim 15 (depends on 12)

The method of claim 12, wherein the first audio embeddings are adapted using an audio grounding adapter trained based on a knowledge-distillation process using a contrastive-learning audio pretrained audio encoder.

### Claim 16 (depends on 12)

The method of claim 12, wherein the first audio embeddings and the second audio embeddings are combined using a cross-attention transformer.

### Claim 17 (depends on 12)

The method of claim 12, wherein the third audio embeddings are projected using a projector trained to generate text projections based on audio embeddings.

### Claim 18 (depends on 12)

The method of claim 12, wherein the audio metadata comprises a list of audio events and corresponding timestamps.

### Claim 19 (depends on 12)

The method of claim 12, wherein the audio metadata is generated based on at least one prior user input.

### Claim 20

A non-transitory computer-readable medium having stored thereon instructions that, when executed by one or more processors, cause the one or more processors to:
encode audio data to generate first audio embeddings;
adapt the first audio embeddings to generate second audio embeddings;
combine the first audio embeddings and the second audio embeddings to generate third audio embeddings;
project the third audio embeddings to generate text projections;
generate audio metadata based on the second audio embeddings; and
process the text projections, the audio metadata, and a query using a machine-learning model to generate a response to the query.
