# US 11,030,259 B2 · Document searching visualized within a document > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,030,259 B2 - Document ID: US11030259B2 - Status: Granted - Kind: B2 - Application number: 15/098,240 - Filed: 2016-04-13 - Published: 2017-10-19 - Granted: 2021-06-08 - Earliest claimed: 2016-04-13 - Assignees: Microsoft Technology Licensing, LLC - Applicants: Microsoft Technology Licensing, LLC - Inventors: Christopher Lee Mullins; Robert Earl Standefer, III; Miguel Oom Temudo de Castro - Web page: https://patentgrep.com/patent/US11030259B2 ## CPC - G06F 16/93: Document management systems - G06F 16/248: Presentation of query results - G06F 16/338: Presentation of query results - G06F 16/9024: {Graphs; Linked lists (G06F16/9027 takes precedence)} - G06F 40/169: Annotation, e.g. comment data or footnotes - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A facility for annotating a visual representation of a document is described. The facility presents on a display device visual representations of two or more portions of a document. The facility further present on the display on the display, in connection with each of one or more of the presented document portion visual representations, an annotation conveying a result of aggregating information describing two or more search transactions in whose results the represented document portion is included. ## Claims ### Claim 1 A persistent storage device having contents that, when executed by a processor, cause the processor to: for a document comprised of document portions: generate a visual representation of at least part of the document containing visual representations of two or more document portions; for each of at least one of the document portions whose visual representations are contained by the document's visual representation: perform an aggregation operation on information describing two or more previous search transactions in whose results the document portion is included to obtain an aggregation result; annotate the document portion's visual representation in the document's visual representation in accordance with the obtained aggregation result; and display the annotated visual representation of the document to be displayed, the annotated visual representation including an indication adjacent to text of the document portion, wherein, for each of the two or more document portions whose visual representations are contained by the document's visual representation, the aggregated search transactions include performed search transactions in whose results the document portion was actually included; wherein the indication includes at least one of a color or a pattern; and wherein the indication relates to a relevance of the document portion on a scale; and wherein, for a selected one of the document portions whose visual representations are contained by the document's visual representation: the information describing the two or more previous search transactions in whose results the document portion is included identifies for each search transaction a query that was the basis for the search transaction, and the annotation includes the content of the query of at least one of the search transactions in whose results the document portion is included, where the query was the basis for the search transaction. ### Claim 2 (depends on 1) The persistent storage device of claim 1 wherein, for a selected one of the document portions whose visual representations are contained by the document's visual representation: the information describing the two or more previous search transactions in whose results the document portion is included identifies for each such search transaction a user who issued a search request upon which the search transaction is based; and the aggregation operation is performed with respect to the user identified for each search transaction. ### Claim 3 (depends on 1) The persistent storage device of claim 1 wherein the document is organized as a graph of nodes connected by edges, and wherein the visual representation of at least part of the document contains visual representations of document portions each corresponding to one of the nodes, and wherein annotating each document portion's visual representation comprises applying a visual quality to the document portion's visual representation in accordance with the aggregation result obtained for the document portion. ### Claim 4 (depends on 3) The persistent storage device of claim 3 wherein, in the visual representation of the document, the edges between the nodes of the graph are explicitly visually represented. ### Claim 5 (depends on 1) The persistent storage device of claim 1 wherein the visual representation of the document contains some or all of the literal contents of at least some of the document portions. ### Claim 6 (depends on 1) The persistent storage device of claim 1 wherein the document is subject to an outline organization made up of outline elements, and wherein each of the document portion visual representations contains one of the outline elements of the outline organization of the document. ### Claim 7 (depends on 1) The persistent storage device of claim 1 wherein the annotations of the document portions' visual representations are selected from among two or more colors. ### Claim 8 (depends on 1) The persistent storage device of claim 1 wherein the performed aggregation operation counts the search transactions in whose results each document portion is included. ### Claim 9 (depends on 1) The persistent storage device of claim 1 wherein, for each of at least one of the document portions whose visual representations are contained by the document's visual representation, the aggregated search transactions additionally include performed search transactions in whose results the document portion was not actually included, but in whose results the document portion would have been included had the document portion been available to the search transaction. ### Claim 10 A processor-based device, comprising: a display; a processor; and a persistent storage device having contents that cause the processor to: present on the display visual representations of two or more portions of a document; and present on the display, in connection with each of one or more of the presented document portion visual representations, an annotation conveying a result of aggregating information describing two or more previous search transactions in whose results the represented document portion is included, wherein the annotation includes an indication adjacent to text of the document portion; wherein, for each of the two or more portions of the document, the result of aggregating information describing two or more previous search transactions include performed search transactions in whose results the document portion was actually included; wherein the indication includes at least one of a color or a pattern; and wherein the indication relates to a relevance of the document portion, the relevance being on a scale; and wherein, for a selected one of the document portions whose visual representations are contained by the document's visual representation: the annotation includes the content of a query of at least one of the search transactions in whose results the document portion is included, where the query was the basis for the search transaction, wherein the annotations of the document portions' visual representations are selected from among two or more colors. ### Claim 11 (depends on 10) The processor-based device of claim 10 wherein, for each of the annotated document portion visual representations the annotation conveys the result of counting the search transactions in whose results the document portion represented by the document portion visual representation is included.