# US 2019/0318009 A1 · Systems and methods for providing feedback for natural language queries > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0318009 A1 - Document ID: US20190318009A1 - Status: Published application - Kind: A1 - Application number: 15/952,896 - Filed: 2018-04-13 - Published: 2019-10-17 - Earliest claimed: 2018-04-13 - Assignees: LexisNexis, a division of Reed Elsevier Inc. - Applicants: LexisNexis, a division of Reed Elsevier Inc. - Inventors: Richard D. Miller; Todd J. Frascone; Jacob Aaron Myers - Web page: https://patentgrep.com/patent/US20190318009A1 ## CPC - G06F 16/24578: {using ranking} - G06F 16/3322: {using system suggestions (G06F16/3325 takes precedence)} - G06F 16/243: {Natural language query formulation} - G06F 16/24575: {using context} - G06F 16/248: Presentation of query results - G06F 16/93: Document management systems - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Systems, methods, and computer-readable media for providing for display an estimated breadth indicator and/or search terms proximity for a result set of documents are disclosed. A method includes receiving a natural language search query including a plurality of search concepts, determining the plurality of search concepts from the natural language query, searching a database using the natural language search query to identify the result set of documents, where the result set of documents are identified based on the plurality of search concepts, calculating a breadth of the result set of documents, where the breadth is calculated from an estimated relevance score that is indicative of a degree to which the result set of documents are relevant to the search query, and providing for display, the breadth as a feedback meter element. The feedback meter element provides a visual indication of the breadth of the natural language search query. ## Claims ### Claim 1 A method for providing for display an estimated breadth indicator for a result set of documents, the method comprising: receiving, at a computing device, a natural language search query comprising a plurality of search concepts; determining the plurality of search concepts from the natural language query; searching a database using the natural language search query to identify the result set of documents, wherein the result set of documents are identified based on the plurality of search concepts; calculating a breadth of the result set of documents, wherein the breadth is calculated from an estimated relevance score that is indicative of a degree to which the result set of documents are relevant to the search query; and providing for display, the breadth as a feedback meter element, wherein the feedback meter element provides a visual indication of the breadth of the natural language search query. ### Claim 2 (depends on 1) The method of claim 1, further comprising: determining one or more context characteristics indicative of a context of the natural language search query, wherein the breadth of the result set of documents is calculated based on the one or more context characteristics. ### Claim 3 (depends on 2) The method of claim 2, wherein the one or more context characteristics includes one or more of: a user profile, a device type, a geographic location, and a network location. ### Claim 4 (depends on 1) The method of claim 1, further comprising: determining one or more query characteristics indicative of a characteristic of the natural language search query, wherein the breadth of the result set of documents is calculated based on the one or more query characteristics. ### Claim 5 (depends on 4) The method of claim 4, wherein the one or more query characteristics include a number of query terms, an inclusion of a recognized phrase, an inclusion of a citation, an inclusion of a filter, or a degree of similarity to at least one previous natural language search query. ### Claim 6 (depends on 1) The method of claim 1, further comprising: determining one or more results characteristics indicative of a characteristic of the result set of documents, wherein the breadth of the result set of documents is calculated based on the one or more results characteristics. ### Claim 7 (depends on 6) The method of claim 6, wherein the one or more results characteristics includes at least one of: a terms relevance indicative of an extent to which query terms match one or more terms in at least one document of the result set of documents, a terms relevance precipitation indicative of a difference in relevance between a first document of the result set of documents and a second document of the result set of documents, a best paragraphs terms prevalence indicative of a number of the query terms and synonyms of the query terms included in at least one paragraph of at least one document of the result set of documents, a topical diversity indicative of a variance in topics within the result set of documents, a terms proximity indicative of a proximity of the query terms within a portion of at least one document of the result set of documents, a query to core terms ratio indicative of a degree of overlap between the query terms and core terms in at least one document of the result set of documents, a document recency indicator indicative of a date of at least one document of the result set of documents, and a number of documents in the result set of documents. ### Claim 8 (depends on 1) The method of claim 1, wherein the breadth is determined based on a plurality of weighted score components. ### Claim 9 (depends on 1) The method of claim 1, further comprising providing one or more suggested search terms for improving the search results or for adjusting the breadth of the result set of documents. ### Claim 10 A method for providing for display an indicator of terms proximity for a result set of documents, the method comprising: receiving, at a computing device, a natural language search query comprising a plurality of search concepts; determining the plurality of search concepts from the natural language query; searching a database using the natural language search query to identify the result set of documents, wherein the result set of documents are identified based on the plurality of search concepts; calculating a proximity of terms within the result set of documents, wherein the proximity of terms is calculated from a coverage of terms in a subset of relevant excerpts from the result set of documents; and providing for display, the proximity of terms as a feedback meter element, wherein the feedback meter element provides a visual indication of the proximity of terms within the result set of documents generated from the natural language search query. ### Claim 11 (depends on 10) The method of claim 10, wherein the coverage of terms represents a percentage of query terms from the natural language query that appear in an excerpt of the subset of relevant excerpts from the result set of documents. ### Claim 12 A system for providing for display an estimated breadth indicator for a result set of documents, the system comprising: a processing device; and a non-transitory, processor-readable storage medium comprising one or more programming instructions thereon that, when executed, cause the processing device to: receive a natural language search query comprising a plurality of search concepts; determine the plurality of search concepts from the natural language query; search a database using the natural language search query to identify the result set of documents, wherein the result set of documents are identified based on the plurality of search concepts; calculate a breadth of the result set of documents, wherein the breadth is calculated from an estimated relevance score that is indicative of a degree to which the result set of documents are relevant to the search query; and provide for display, the breadth as a feedback meter element, wherein the feedback meter element provides a visual indication of the breadth of the natural language search query. ### Claim 13 (depends on 12) The system of claim 12, wherein the one or more programming instructions, when executed, further cause the processing device to: determine one or more context characteristics indicative of a context of the natural language search query, wherein the breadth of the result set of documents is calculated based on the one or more context characteristics. ### Claim 14 (depends on 13) The system of claim 13, wherein the one or more context characteristics includes one or more of: a user profile, a device type, a geographic location, and a network location. ### Claim 15 (depends on 12) The system of claim 12, wherein the one or more programming instructions, when executed, further cause the processing device to: determine one or more query characteristics indicative of a characteristic of the natural language search query, wherein the breadth of the result set of documents is calculated based on the one or more query characteristics. ### Claim 16 (depends on 15) The system of claim 15, wherein the one or more query characteristics include a number of query terms, an inclusion of a recognized phrase, an inclusion of a citation, an inclusion of a filter, or a degree of similarity to at least one previous natural language search query. ### Claim 17 (depends on 12) The system of claim 12, wherein the one or more programming instructions, when executed, further cause the processing device to: determine one or more results characteristics indicative of a characteristic of the result set of documents, wherein the breadth of the result set of documents is calculated based on the one or more results characteristics. ### Claim 18 (depends on 17) The system of claim 17, wherein the one or more results characteristics includes at least one of: a terms relevance indicative of an extent to which query terms match one or more terms in at least one document of the result set of documents, a terms relevance precipitation indicative of a difference in relevance between a first document of the result set of documents and a second document of the result set of documents, a best paragraphs terms prevalence indicative of a number of the query terms and synonyms of the query terms included in at least one paragraph of at least one document of the result set of documents, a topical diversity indicative of a variance in topics within the result set of documents, a terms proximity indicative of a proximity of the query terms within a portion of at least one document of the result set of documents, a query to core terms ratio indicative of a degree of overlap between the query terms and core terms in at least one document of the result set of documents, a document recency indicator indicative of a date of at least one document of the result set of documents, and a number of documents in the result set of documents. ### Claim 19 (depends on 12) The system of claim 12, wherein the breadth is determined based on a plurality of weighted score components. ### Claim 20 (depends on 12) The system of claim 12, wherein the one or more programming instructions, when executed, further cause the processing device to: provide one or more suggested search terms for improving the search results or for adjusting the breadth of the result set of documents.