Dynamic similarity threshold selection for natural language caches
AssigneeCisco Technology, Inc.
InventorsArun Kwangil Iyengar, Ashish Kundu
Life of this patent
Estimating the term…
Abstract
A device receives a query for input to a language model. The device then selects a particular similarity threshold based on information associated with the query. The device makes, using the particular similarity threshold, a determination as to whether the query matches a cached query. The device provides, based on the determination, a response associated with the cached query in lieu of inputting the query to the language model.
Claims20
A method comprising:
- receiving, at a device, a query for input to a language model;
- selecting, by the device, a particular similarity threshold based on information including at least one of:
- a user preference, a query type, a latency for receiving at least one response from the language model, a cost to make a query to the language model, or a level of network connectivity with the language model;
- making, by the device and using the particular similarity threshold, a determination as to whether the query matches a cached query for which the language model previously issued a response;
- and
- providing, by the device and based on the determination, the response associated with the cached query in lieu of inputting the query to the language model.
An apparatus, comprising:
- one or more network interfaces;
- a processor coupled to the one or more network interfaces and configured to execute one or more processes;
- and
- a memory configured to store a process that is executable by the processor, the process when executed configured to:
- receive a query for input to a language model;
- select a particular similarity threshold based on information associated with the query;
- make, using the particular similarity threshold, a determination as to whether the query matches a cached query for which the language model previously issued a response;
- and
- provide, based on the determination, the response associated with the cached query in lieu of inputting the query to the language model.
A method for improving response times to a query answering service, wherein the query answering service provides natural language responses to queries, the method comprising steps of:
- storing the queries made to the query answering service and corresponding responses from the query answering service in a cache;
- determining a semantic similarity threshold using at least one of a user preference, a query type, a latency for receiving at least one response from the query answering service, a cost to make a query to the query answering service, or a level of network connectivity with the query answering service, wherein the semantic similarity threshold is correlated with a level of semantic similarity between two natural language texts;
- receiving a query q1 at a device;
- determining a semantic similarity between the query q1 and at least one query q2 stored in the cache for which the query answering service previously issued a response r1;
- and
- in response to determining at least one query q2 stored in the cache for which the semantic similarity between the query q1 and the at least one query q2 is greater than or equal to the semantic similarity threshold, returning the response r1 stored in the cache corresponding to the at least one query q2.
Dependencies are inferred from the claim text.