# US 10,769,186 B2 · System and method for contextual reasoning > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,769,186 B2 - Document ID: US10769186B2 - Status: Granted - Kind: B2 - Application number: 15/785,347 - Filed: 2017-10-16 - Published: 2019-04-18 - Granted: 2020-09-08 - Earliest claimed: 2017-10-16 - Assignees: Nuance Communications, Inc. - Applicants: Nuance Communications, Inc. - Inventors: Peter Yeh; Ezra Story; Prateek Jain - Web page: https://patentgrep.com/patent/US10769186B2 ## CPC - G06F 16/3329: Natural language query formulation - G06F 16/24575: {using context} - G06N 5/04: Inference or reasoning models - G06N 5/043: Distributed expert systems; Blackboards - G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks - G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue - G06N 5/046: Forward inferencing; Production systems - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING ## Abstract In an embodiment, a method includes determining, based on a received query and contextual information, candidate reasoners to respond to a received query to select a candidate reasoner. A reasoner or candidate reasoner is a module that translates information from a sensor, user settings, or other source, into additional or revised fields for a query. The method further includes generating, at each candidate reasoner determined, additional or revised query fields based on the contextual information and a rule of a rule database. The method further includes merging the additional query fields for each candidate reasoner based on a confidence score or other metric of each corresponding candidate reasoner. The confidence score can be based on applicability of the contextual information to the received query. The method further includes providing an enhanced query having the additional or revised query fields. Such a method enhances natural language understanding by supplementing queries with contextual information. ## Claims ### Claim 1 A method of processing queries at an interactive voice response system, the method comprising: determining, based on a received query and contextual information, a plurality of candidate reasoners and a dimension of each candidate reasoner, each candidate reasoner having capabilities of multiple dimensions of information within each capability, to respond to at least a portion of the received query by (a) determining capabilities and dimensions needed by a candidate reasoner to resolve the received query and (b) matching the capabilities and dimensions determined to capabilities and dimensions associated with each of the plurality of candidate reasoners; generating, at each candidate reasoner determined, at least one additional or revised query field directed to the dimension determined or value based on the contextual information, the received query, and a rule of a rule database; merging the at least one additional or revised query field or value for each candidate reasoner based on a confidence score associated with a result of each corresponding candidate reasoner, the confidence score based on applicability of the contextual information to the received query; and providing an enhanced query to an intelligent knowledge layer for retrieving data from a knowledge repository able to process information related to the determined dimension, the enhanced query having the at least one additional or revised query field or value. ### Claim 2 (depends on 1) The method of claim 1, further comprising: determining whether the at least one additional or revised query field or value should be input to at least one candidate reasoner to generate further additional or revised query fields or values. ### Claim 3 (depends on 1) The method of claim 1, further comprising: generating the received query based on a natural language understanding request. ### Claim 4 (depends on 1) The method of claim 1, wherein a plurality of the candidate reasoners can be configured to be considered, the plurality of candidate reasoners including at least one of a spatial reasoning module configured to perform reasoning based on locations of information in the query, a temporal reasoning module configured to perform reasoning based on timing of information in the query, contextual reasoning configured to perform reasoning based on contextual information, preference reasoning configured to perform reasoning based on preferences of the users, and custom reasoning configured to perform reasoning based on domain specific information present in the context or received query. ### Claim 5 (depends on 1) The method of claim 1, further comprising: gathering contextual information from sensors of a system, the system being separate from the interactive voice response system, receiving the query. ### Claim 6 (depends on 1) The method of claim 1, wherein merging the at least one additional or revised query fields or values further includes prioritizing at least one additional or revised query fields or values from a reasoner ranked with a higher priority. ### Claim 7 (depends on 1) The method of claim 1, wherein the determining, generating, merging, and providing are performed by a reasoning interface layer, the method further comprising: providing the enhanced query to an intelligent knowledge layer configured to resolve each of the additional query fields of the enhanced query. ### Claim 8 (depends on 1) The method of claim 1, further comprising: determining, based on the enhanced query a plurality of sources, a source being an external content provider, a virtual assistant, or internal database, that can be used to fulfill all or a portion of the enhanced query; and constructing a plan, the plan indicating which sources to invoke and the order to invoke the sources. ### Claim 9 A system for processing queries at an interactive voice response system, the system comprising: a processor; and a memory with computer code instructions stored therein, the memory operatively coupled to said processor such that the computer code instructions configure the processor to implement a reasoning interface layer that is configured to: determine, based on a received query and contextual information, a plurality of candidate reasoners and a dimension of each candidate reasoner, each candidate reasoner having capabilities of multiple dimensions of information within each capability, to respond to at least a portion of the received query by (a) determining capabilities and dimensions needed by a candidate reasoner to resolve the received query and (b) matching the capabilities and dimensions determined to capabilities and dimensions associated with each of the plurality of candidate reasoners; generate, at each candidate reasoner determined, at least one additional or revised query field directed to the dimension determined or value based on the contextual information, the received query, and a rule of a rule database; merge the at least one additional or revised query field or value for each candidate reasoner based on a confidence score associated with a result of each corresponding candidate reasoner, the confidence score based on applicability of the contextual information to the received query; and provide an enhanced query to an intelligent knowledge layer for retrieving data from a knowledge repository able to process information related to the determined dimension, the enhanced query having the at least one additional or revised query field or value. ### Claim 10 (depends on 9) The system of claim 9, wherein the reasoning interface layer is further configured to: determine whether the at least one additional or revised query field or value should be input to at least one candidate reasoner to generate further additional or revised query fields or values. ### Claim 11 (depends on 9) The system of claim 9, wherein the reasoning interface layer is further configured to: generate the received query based on a natural language understanding request. ### Claim 12 (depends on 9) The system of claim 9, wherein a plurality of the candidate reasoners can be configured to be considered, the plurality of candidate reasoners configured to perform reasoning based on at least one of locations of information in the query, timing of information in the query, contextual information, preferences of the users, and domain specific information present in the context or received query. ### Claim 13 (depends on 9) The system of claim 9, wherein the reasoning interface layer is further configured to: receive contextual information from sensors of a system, the system separate from the interactive voice response system, receiving the query. ### Claim 14 (depends on 9) The system of claim 9, wherein merging the at least one additional or revised query fields or values further includes prioritizing at least one additional or revised query fields or values from a reasoner ranked with a higher priority. ### Claim 15 (depends on 9) The system of claim 9, wherein the reasoning interface layer is further configured to: providing the enhanced query to an intelligent knowledge layer configured to resolve each of the additional query fields of the enhanced query. ### Claim 16 (depends on 9) The system of claim 9, wherein the reasoning interface layer is further configured to: determine, based on the enhanced query a plurality of sources, a source being an external content provider, a virtual assistant, or internal database, that can be used to fulfill all or a portion of the enhanced query; and construct a plan, the plan indicating which sources to invoke and the order to invoke the sources. ### Claim 17 A non-transitory computer-readable medium configured to store instructions for processing queries at an interactive voice response system, the instructions, when loaded and executed by a processor, causes the processor to: determine, based on a received query and contextual information, a plurality of candidate reasoners and a dimension of each candidate reasoner, each candidate reasoner having capabilities of multiple dimensions of information, to respond to at least a portion of the received query by (a) determining capabilities and dimensions needed by a candidate reasoner to resolve the received query and (b) matching the capabilities and dimensions determined to capabilities and dimensions associated with each of the plurality of candidate reasoners; generate, at each candidate reasoner determined, at least one additional or revised query field directed to the dimension determined or value based on the contextual information, the received query, and a rule of a rule database; merge the at least one additional or revised query field or value for each candidate reasoner based on a confidence score associated with a result of each corresponding candidate reasoner, the confidence score based on applicability of the contextual information to the received query; and provide an enhanced query to an intelligent knowledge layer for retrieving data from a knowledge repository able to p rocess information related to the determined dimension, the enhanced query having the at least one additional or revised query field or value. ### Claim 18 (depends on 17) The non-transitory computer readable medium of claim 17, wherein the instructions further cause the processor to: determine whether the at least one additional or revised query field or value should be input to at least one candidate reasoner to generate further additional or revised query fields or values. ### Claim 19 (depends on 17) The non-transitory computer readable medium of claim 17, wherein the instructions further cause the processor to: generate the received query based on a natural language understanding request. ### Claim 20 (depends on 17) The non-transitory computer readable medium of claim 17, wherein a plurality of the candidate reasoners can be configured to be considered, the plurality of candidate reasoners including at least one of a spatial reasoning module configured to perform reasoning based on locations of information in the query, a temporal reasoning module configured to perform reasoning based on timing of information in the query, contextual reasoning configured to perform reasoning based on contextual information, preference reasoning configured to perform reasoning based on preferences of the users, and custom reasoning configured to perform reasoning based on domain specific information present in the context or received query.