# US 12,602,198 B2 · Search and knowledge base question answering for a voice user interface

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,602,198 B2
- Document ID: US12602198B2
- Status: Granted
- Kind: B2
- Application number: 18/215,872
- Filed: 2023-06-29
- Published: 2024-02-29
- Granted: 2026-04-14
- Earliest claimed: 2017-05-30
- Assignees: Amazon Technologies, Inc.
- Applicants: Amazon Technologies, Inc.
- Inventors: Daniel Lewis Spector; Fergus O′Donoghue; Chase Wesley Brown, Jr.; Shayne Leon Snow; Brandon Gerald Li Horst; William Folwell Barton
- Web page: https://patentgrep.com/patent/US12602198B2

## CPC

- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G06F 16/3329: Natural language query formulation
- G06F 16/951: Indexing; Web crawling techniques
- G10L 15/1815: {Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning}
- G10L 15/26: Speech to text systems (G10L15/08 takes precedence)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

A voice-controlled question answering system that is capable of answering questions using both a knowledge base and a search engine. The knowledge base is used to answer questions when answers to those questions are contained in the knowledge base. If an answer using the knowledge base is unavailable, and if the question is suitable for answering using an unstructured search approach, the system may obtain an answer using a search engine. The search engine results may be processed to obtain an answer to the question suitable for output using a voice user interface.

## Claims

### Claim 1

A computer-implemented method, comprising:
receiving first data corresponding to a natural language input user query;
processing the first data using at least one machine learning model to determine model output data;
determining, based at least in part on the model output data, to perform a search corresponding to the natural language input user query;
sending data representing the natural language input user query to a search component;
receiving, from the search component, response data; and
causing an output using the response data.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, wherein the at least one machine learning model comprises a classifier.

### Claim 3 (depends on 1)

The computer-implemented method of claim 1, wherein the at least one machine learning model is trained using text data representing prior user queries.

### Claim 4 (depends on 1)

The computer-implemented method of claim 1, wherein the model output data indicates a question type corresponding to the natural language input user query.

### Claim 5 (depends on 1)

The computer-implemented method of claim 1, wherein the model output data indicates a question topic corresponding to the natural language input user query.

### Claim 6 (depends on 1)

The computer-implemented method of claim 1, wherein causing the output using the response data comprises:
processing the response data based at least in part on the natural language input user query to determine a first portion of the response data that represents a potential response to the natural language input user query;
determining natural language data corresponding to the first portion of the response data; and
causing the output representing the natural language data.

### Claim 7 (depends on 1)

The computer-implemented method of claim 1, where causing the output using the response data comprises:
processing the response data using a text-to-speech component to determine audio data; and
causing playback of audio using the audio data.

### Claim 8 (depends on 1)

The computer-implemented method of claim 1, further comprising:
determining that a first component is not capable of determining an answer to the natural language input user query,
wherein determining to perform the search is further based at least in part on the first component not being capable of determining the answer.

### Claim 9 (depends on 1)

The computer-implemented method of claim 1, wherein the model output data comprises confidence data.

### Claim 10 (depends on 1)

The computer-implemented method of claim 1, further comprising:
determining a source corresponding to the response data; and
including, within the output, an indication corresponding to the source.

### Claim 11

A system comprising:
at least one processor; and
at least one memory comprising instructions that, when executed by the at least one processor, cause the system to:
receive first data corresponding to a natural language input user query;
process the first data using at least one machine learning model to determine model output data;
determine, based at least in part on the model output data, to perform a search corresponding to the natural language input user query;
send data representing the natural language input user query to a search component;
receive, from the search component, response data; and
cause an output using the response data.

### Claim 12 (depends on 11)

The system of claim 11, wherein the at least one machine learning model comprises a classifier.

### Claim 13 (depends on 11)

The system of claim 11, wherein the at least one machine learning model is trained using text data representing prior user queries.

### Claim 14 (depends on 11)

The system of claim 11, wherein the model output data indicates a question type corresponding to the natural language input user query.

### Claim 15 (depends on 11)

The system of claim 11, wherein the model output data indicates a question topic corresponding to the natural language input user query.

### Claim 16 (depends on 11)

The system of claim 11, wherein the instructions that cause the system to cause the output using the response data comprise instructions that, when executed by the at least one processor, cause the system to:
process the response data based at least in part on the natural language input user query to determine a first portion of the response data that represents a potential response to the natural language input user query;
determine natural language data corresponding to the first portion of the response data; and
cause the output representing the natural language data.

### Claim 17 (depends on 11)

The system of claim 11, wherein the instructions that cause the system to cause the output using the response data comprise instructions that, when executed by the at least one processor, cause the system to:
process the response data using a text-to-speech component to determine audio data; and
cause playback of audio using the audio data.

### Claim 18 (depends on 11)

The system of claim 11, wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:
determine that a first component is not capable of determining an answer to the natural language input user query,
wherein determining to perform the search is further based at least in part on the first component not being capable of determining the answer.

### Claim 19 (depends on 11)

The system of claim 11, wherein the model output data comprises confidence data.

### Claim 20 (depends on 11)

The system of claim 11, wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:
determine a source corresponding to the response data; and
include, within the output, an indication corresponding to the source.