# US 2023/0267126 A1 · Caching Historical Embeddings in Conversational Search

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0267126 A1
- Document ID: US20230267126A1
- Status: Published application
- Kind: A1
- Application number: 18/172,099
- Filed: 2023-02-21
- Published: 2023-08-24
- Earliest claimed: 2022-02-23
- Assignees: Georgetown University
- Applicants: Georgetown University
- Inventors: Ophir Frieder; Ida Mele; Christina-Ioana Muntean; Franco Maria Nardini; Raffaele Perego; Nicola Tonellotto
- Web page: https://patentgrep.com/patent/US20230267126A1

## CPC

- G06F 16/24578: {using ranking}
- G06F 16/3349: Reuse of stored results of previous queries
- G06F 16/243: {Natural language query formulation}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A method and system are described for improving the speed and efficiency of obtaining conversational search results. A user may speak a phrase to perform a conversational search or a series of phrases to perform a series of searches. These spoken phrases may be enriched by context and then converted into a query embedding. A similarity between the query embedding and document embeddings is used to determine the search results including a query cutoff number of documents and a cache cutoff number of documents. A second search phrase may use the cache of documents along with comparisons of the returned documents and the first query embedding to determine the quality of the cache for responding to the second search query. If the results are high-quality then the search may proceed much more rapidly by applying the second query only to the cached documents rather than to the server.

## Claims

### Claim 1

A method for improving the speed and efficiency of obtaining search results comprising the steps of:
receiving a first utterance at a server;
converting the first utterance into a first query in an embedding space;
performing, at the server, a first conversational search by:
comparing a first similarity of the first query with an embedding of each document of a plurality of documents;
returning a cache cutoff number of documents ranked by the first similarity; and
returning a query cutoff number of documents ranked by the first similarity;
storing as a cache, on a local device, the first query and the cache cutoff number of documents ranked by the first similarity;
receiving a second utterance at the server;
converting the second utterance into a second query in the embedding space;
determining, at the local device, using the second query and the cache whether the second query applied to the cache documents provides low quality results or high quality results;
if the second query applied to the cache provides low quality results, then performing a second conversational search on the server by:
calculating a second similarity of the second query with an embedding of each document of the plurality of documents; and
returning the query cutoff number of documents ranked by the second similarity;
if the second query applied to the cache provides high quality results, then performing, at the local device, the second conversational search by:
calculating a second similarity of the second query with an embedding of each document of the cache; and
returning the query cutoff number of documents ranked by the second similarity;
reporting the query cutoff number of documents ranked by the first similarity to the local device; and
reporting the query cutoff number of documents ranked by the second similarity to the local device.

### Claim 2 (depends on 1)

The method of claim 1, wherein the cache cutoff number is greater than the query cutoff number.

### Claim 3 (depends on 1)

The method of claim 1, wherein the first utterance and the second utterance are semantically enriched by context prior to being converted into the first query and the second query.

### Claim 4 (depends on 1)

The method of claim 1, wherein determining whether the second query applied to the cache provides low quality results or high-quality results comprises:
determining a query distance in embedding space between the first query and the second query;
determining a search distance in embedding space between the first query and the least similar document of the returned cache cutoff number of documents from the first conversational search; and
comparing the query distance with the search distance:
if the query distance is less than or equal to the search distance, then the second query applied to the cache provides high quality results;
if the query distance is greater than the search distance, then the second query applied to the cache provides low quality results.

### Claim 5 (depends on 4)

The method of claim 4, wherein comparing the query distance with the search distance further comprises:
if the query distance plus a hyperparameter is less than or equal to the search distance, then the second query applied to the cache provides high quality results;
if the query distance plus the hyperparameter is greater than the search distance, then the second query applied to the cache provides low quality results.

### Claim 6 (depends on 5)

The method of claim 5, wherein the hyperparameter is selected based on a user preference.

### Claim 7 (depends on 1)

The method of claim 1, wherein the second conversational search is optimized taking into account at least one of the following evaluation metrics: a hit rate, an average query response time, a mean average precision at query cutoff 200, a mean reciprocal rank at query cutoff 200, a normalized discounted cumulative gain at query cutoff 3, a precision at query cutoff 1, a precision at query cutoff 3, and a coverage of a query with respect to the local cache and the query cutoff number.

### Claim 8 (depends on 1)

The method of claim 1, wherein the local device comprises at least one of a mobile phone, a smart speaker, an internet of things device, a personal computer, or a laptop computer.

### Claim 9

A system for improving the speed and efficiency of obtaining search results comprising:
a local device for receiving a first utterance and a second utterance and transmitting the first utterance and the second utterance to a server;
the server and the local device to perform a method comprising:
converting, at the server, the first utterance into a first query in an embedding space;
performing, at the server, a first conversational search by:
comparing a first similarity of the first query with an embedding of each document of a plurality of documents;
returning a cache cutoff number of documents ranked by the first similarity; and
returning a query cutoff number of documents ranked by the first similarity;
storing as a cache, on the local device, the first query and the cache cutoff number of documents ranked by the first similarity;
converting, at the server, the second utterance into a second query in the embedding space;
determining, at the local device, using the second query and the cache whether the second query applied to the cache documents provides low-quality results or high-quality results;
if the second query applied to the cache provides low-quality results, then performing a second conversational search on the server by:
calculating a second similarity of the second query with an embedding of each document of the plurality of documents; and
returning the query cutoff number of documents ranked by the second similarity;
if the second query applied to the cache provides high-quality results, then performing, at the local device, the second conversational search by:
calculating a second similarity of the second query with an embedding of each document of the cache; and
returning the query cutoff number of documents ranked by the second similarity;
reporting the query cutoff number of documents ranked by the first similarity to the local device; and
reporting the query cutoff number of documents ranked by the second similarity to the local device.

### Claim 10 (depends on 9)

The system of claim 9, wherein the cache cutoff number is greater than the query cutoff number.

### Claim 11 (depends on 9)

The system of claim 9, wherein the first utterance and the second utterance are semantically enriched by context prior to being converted into the first query and the second query.

### Claim 12 (depends on 9)

The system of claim 9, wherein determining whether the second query applied to the cache provides low-quality results or high-quality results comprises:
determining a query distance in embedding space between the first query and the second query;
determining a search distance in embedding space between the first query and the least similar document of the returned cache cutoff number of documents from the first conversational search; and
comparing the query distance with the search distance:
if the query distance is less than or equal to the search distance, then the second query applied to the cache provides high-quality results;
if the query distance is greater than the search distance, then the second query applied to the cache provides low-quality results.

### Claim 13 (depends on 12)

The system of claim 12, wherein comparing the query distance with the search distance further comprises:
if the query distance plus a hyperparameter is less than or equal to the search distance, then the second query applied to the cache provides high-quality results;
if the query distance plus the hyperparameter is greater than the search distance, then the second query applied to the cache provides low-quality results.

### Claim 14 (depends on 13)

The system of claim 13, wherein the hyperparameter is selected based on a user preference.

### Claim 15 (depends on 9)

The system of claim 9, wherein the second conversational search is optimized taking into account at least one of the following evaluation metrics: a hit rate, an average query response time, a mean average precision at query cutoff 200, a mean reciprocal rank at query cutoff 200, a normalized discounted cumulative gain at query cutoff 3, a precision at query cutoff 1, a precision at query cutoff 3, and a coverage of a query with respect to the local cache and the query cutoff number.

### Claim 16 (depends on 9)

The system of claim 9, wherein the local device comprises at least one of a mobile phone, a smart speaker, an internet of things device, a personal computer, or a laptop computer.
