# US 12,182,206 B2 · User context-based enterprise search with multi-modal interaction > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,182,206 B2 - Document ID: US12182206B2 - Status: Granted - Kind: B2 - Application number: 17/385,802 - Filed: 2021-07-26 - Published: 2023-01-26 - Granted: 2024-12-31 - Earliest claimed: 2021-07-26 - Assignees: Microsoft Technology Licensing, LLC - Applicants: Microsoft Technology Licensing, LLC - Inventors: Ming Wu; Yue Ma; Yong Ni - Web page: https://patentgrep.com/patent/US12182206B2 ## CPC - G06F 16/90335: {Query processing} - G06F 16/3329: Natural language query formulation - G06F 16/90332: {Natural language query formulation or dialogue systems} - G06F 16/9038: Presentation of query results - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Examples of the present disclosure describe systems and methods for enterprise search that leverage periodically updated user context of an enterprise user for intent understanding and treat a search session as a dialog between the user and a digital assistant to allow multi-modal interaction. For example, a query input by the user may be received from a client application having search functionality and an integrated assistant. A current state of the user context may be leveraged to understand the query. Based on the understanding, one or more responsive entities may be retrieved from an enterprise search index as results. Based on the results, a response may be generated that includes the results and a prompt to cause the user to further refine the query and/or provide feedback. The response may be provided to the client application for output as part of a search session dialog between the user and assistant. ## Claims ### Claim 1 A system comprising: at least one processor; and at least one memory coupled to the at least one processor and storing instructions that, when executed by the at least one processor, cause the system to: receive, from a client application having search functionality and an integrated digital assistant of an enterprise, an enterprise search query indicative of a user question asking the integrated digital assistant to retrieve an enterprise file, the enterprise search query initiating or continuing a search session dialog with the integrated digital assistant, the enterprise being a business organization, the enterprise file being accessible only by one or more employees of the business organization based at least in part on a permission setting of the client application; for an enterprise user, periodically update a user context based on at least a plurality of past enterprise search queries prior to the enterprise search query; determine an intent and one or more slots of the enterprise search query based in part on the past enterprise search queries and the enterprise search query, wherein the intent indicates the enterprise user's intention for the integrated digital assistant to retrieve the enterprise file, and wherein the one or more slots specify a file type identifier of the enterprise file; based at least in part on: the intent that indicates the enterprise user's intention for the integrated digital assistant to retrieve the enterprise file, the enterprise file being accessible only by one or more employees of the business organization, and the one or more slots specifying the file type identifier of the enterprise file, retrieve the enterprise file via an enterprise search index that indicates a plurality of enterprise files stored to one or more data sources of the business organization; based at least in part on the retrieving of the enterprise file via the enterprise search index that indicates the plurality of enterprise files, generate a response to the user question asking the integrated digital assistant to retrieve the enterprise file, wherein the response includes a prompt in natural language asking whether the enterprise file is a correct enterprise file; and provide the response to the client application for output as part of the search session dialog. ### Claim 2 (depends on 1) The system of claim 1, wherein the enterprise search query comprises a plurality of terms explicitly expressed in the enterprise search query, and to determine the intent and the one or more slots of the enterprise search query based in part on the user context for the enterprise user, the system is caused to: using one or more natural language processing techniques, determine the intent and at least one slot of the one or more slots based on an analysis of the plurality of terms explicitly expressed in the enterprise search query, wherein the plurality of terms are further analyzed based in part on the current state of the user context. ### Claim 3 (depends on 2) The system of claim 2, wherein the system is further caused to: determine one or more additional slots that are not associated with the plurality of terms explicitly expressed in the enterprise search query based on the user context for the enterprise user to further refine the enterprise search query. ### Claim 4 (depends on 1) The system of claim 1, wherein the data from the plurality of past enterprise search queries includes, for each query, one or more of data associated with the query, detected intent and slots, retrieved results, generated responses, and user interactions to the responses. ### Claim 5 (depends on 1) The system of claim 1, wherein the system is further caused to: receive, from the client application, one or more interactions of the enterprise user with the response, the one or more interactions including at least one of a further refinement of the enterprise search query and feedback related to the results, wherein the user context is updated based on the one or more interactions. ### Claim 6 (depends on 1) The system of claim 1, wherein the user context is periodically updated further based on at least one of: history associated with the digital assistant, action history, settings of the client application, entity preferences, and personal grammars. ### Claim 7 (depends on 1) The system of claim 1, wherein the system is further caused to: infer intent-based user centric signals from the current state of the user context. ### Claim 8 (depends on 7) The system of claim 7, wherein the intent-based user centric signals include a plurality of features, and each of the plurality of features includes one or more values associated with a weight. ### Claim 9 (depends on 7) The system of claim 7, wherein the system is further caused to: determine the intent and the one or more slots of the enterprise search query based in part on the intent-based user centric signals. ### Claim 10 (depends on 7) The system of claim 7, wherein the one or more of the plurality of entities retrieved from the enterprise search index as the results include at least two entities, and the system is further caused to: rank the at least two entities within the results based in part on an associated weight from the intent-based user centric signals. ### Claim 11 (depends on 1) The system of claim 1, wherein: the search session dialog includes a multi-turn query, wherein each turn of the multi-turn query comprises a query that is input by the enterprise user and a response to the query that is output by the digital assistant, the enterprise search query and the response comprise a first turn of the search session dialog, and subsequent to the providing of the response, the system is further caused to: receive, from the client application, another enterprise search query input by the enterprise user as part of a second turn of the search session dialog, wherein an intent and one or more slots of the other enterprise search query are determined based in part on a current state of the user context for the enterprise user and a search session context gleaned from the first turn of the search session dialog. ### Claim 12 (depends on 1) The system of claim 1, wherein the enterprise search query includes a command for a task to be performed by the digital assistant. ### Claim 13 (depends on 1) The system of claim 1, wherein a plurality of input modes are provided, via the client application, to the enterprise user to enable interaction with the search session dialog, the plurality of input modes including at least voice, touch, and text input modes. ### Claim 14 (depends on 1) The system of claim 1, wherein the plurality of entities stored in the enterprise search index include files, electronic communications, contacts, and meetings retrieved from a plurality of enterprise data sources. ### Claim 15 A computer-implemented method comprising: for an enterprise user, periodically updating a user context for enterprise search based on at least a plurality of past enterprise search queries, the enterprise being a business organization, the enterprise user being an employee of the enterprise; receiving, from a client application having search functionality and an integrated digital assistant and as part of a search session dialog, an enterprise search query indicative of a user question asking the integrated digital assistant to retrieve an enterprise file, the enterprise file being accessible only by one or more members of the business organization based at least in part on a permission setting of the client application; determining an intent and one or more slots of the enterprise search query based in part on the plurality of past enterprise search queries and the enterprise search query, wherein the intent indicates the enterprise user's intention for the integrated digital assistant to retrieve the enterprise file and the one or more slots indicate specify a file type identifier of the enterprise file indicated in the enterprise search query; based at least in part on: the determining of the intent that indicates the enterprise user's intention for the integrated digital assistant to retrieve the enterprise file, the permission setting, and the one or more slots specifying the file type indicated in the enterprise search query, retrieving, a plurality of files of the business organization; based at least in part on the retrieving of the plurality of files, generating a response that includes a prompt in natural language asking which file, of the plurality of files the user wants; and providing the response to the client application for output as part of the search session dialog. ### Claim 16 (depends on 15) The computer-implemented method of claim 15, wherein the enterprise search query includes a plurality of terms explicitly expressed in the enterprise search query, and determining the intent and the one or more slots based in part on a current state of the user context for the enterprise user comprises: using one or more natural language processing techniques, determining the intent and at least one slot of the one or more slots based on an analysis of the plurality of terms explicitly expressed in the enterprise search query, wherein the plurality of terms are further analyzed based in part on the current state of the user context; and determining one or more additional slots that are not associated with the plurality of terms explicitly expressed in the enterprise search query based on the current state of the user context for the enterprise user to further refine the enterprise search query. ### Claim 17 (depends on 1) The system of claim 1, wherein the system is further caused to: receive, from the client application, one or more interactions of the enterprise user with the response, the one or more interactions including at least one of a further refinement of the enterprise search query and feedback related to the results, wherein the user context is updated based on one or more of data associated with the enterprise search query, the intent and the one or more slots, the results, the response, and the one or more interactions. ### Claim 18 (depends on 15) The computer-implemented method of claim 15, further comprising: inferring intent-based user centric signals from a current state of the user context; and determining the intent and the one or more slots of the enterprise search query based in part on the intent-based user centric signals. ### Claim 19 (depends on 15) The computer-implemented method of claim 15, wherein: the search session dialog includes a multi-turn query, wherein each turn of the multi-turn query comprises a query that is input by the enterprise user and a response to the query that is output by the digital assistant, the enterprise search query and the response comprise a first turn of the search session dialog, and subsequent to the providing of the response, the method further comprises: receiving, from the client application, an other enterprise search query input by the enterprise user as part of a second turn of the search session dialog; and determining an intent and one or more slots of the other enterprise search query based in part on the current state of the user context for the enterprise user and a search session context gleaned from the first turn of the search session dialog. ### Claim 20 Computer storage media storing instructions, that when executed by a processor, causes the processor to perform operations comprising: for an enterprise user, periodically updating a user context for enterprise search at least a plurality of past enterprise search queries the enterprise user being an employee of a business organization; receiving, from a client application having search functionality and an integrated digital assistant, an enterprise search query indicative of a user question asking the integrated digital assistant to retrieve an enterprise file and input by the enterprise user, the input of the enterprise search query initiating or continuing a search session dialog with the digital assistant, the enterprise search query being submitted by the enterprise user subsequent to the plurality of past enterprise search queries, the enterprise file being accessible only by one or more members of the business organization based at least in part on a permission setting of the client application; determining an intent and one or more slots of the enterprise search query based in part on the plurality of past enterprise search queries, wherein the intent indicates the enterprise user's intention for the integrated digital assistant to retrieve the enterprise file and the one or more slots specify a file type identifier of the enterprise file; based at least in part on: the determining of the intent, the permission setting, and the one or more slots specifying the file type, retrieving, from an enterprise search index comprising a plurality of files, the enterprise file; based at least in part on the retrieving the enterprise file from the enterprise search index comprising the plurality of files, generating a response, by the integrated digital assistant, that includes a link to the enterprise file; and providing the response to the client application for output to the enterprise user as part of the search session dialog.